> If you know what you’re doing and you know where to fit the LLMs (they’re genuinely the best system we’ve ever devised for interpreting and categorizing unstructured human input), they can be immensely useful, but they sing a siren song of simplicity that will lure you to your doom if you believe it.
I imagine using their embeddings and training a classifier on top of that is probably a lot more effective?
I've personally found agentic LLM workflows the most effective as extremely sophisticated autocomplete. Instead of autocompleting the current next few tokens, I tell it precisely how to edit my code at a high level. You can't tell it stuff at a feature level, but telling it how to implement the feature saves me a ton of time.
> I imagine using their embeddings and training a classifier on top of that is probably a lot more effective?
I’d be interested in seeing this in action - I think the vector embeddings are underused generally - but my understanding is that’d be for something closer to sentiment analysis? In this case I’m talking about a setup closer to where you’ve got an LLM agent with a set of tools that’s interpreting a user’s request to identify which of those tools are the right ones to use. The requests can be complex, and involve multiple tool runs or chaining. If that’s doable by more deterministic mechanisms, I’d (genuinely) love to hear about it.
If you're only working on one problem that's very valuable to solve, then taking the time to train a classifier is great.
The beauty of LLMs is that you can run a ton of experiments, notebooks, demos etc because you can write classifiers and structure unstructured data so fast, in a reasonably accurate way (at the moment it seems roughly in line with say hiring an intern to label things)
I imagine using their embeddings and training a classifier on top of that is probably a lot more effective?
I've personally found agentic LLM workflows the most effective as extremely sophisticated autocomplete. Instead of autocompleting the current next few tokens, I tell it precisely how to edit my code at a high level. You can't tell it stuff at a feature level, but telling it how to implement the feature saves me a ton of time.