> When I save twenty hours of a client's money and my own time, by telling them that a new software feature they want would be unnecessary if they changed the order of questions their employees ask on the phone, I've done my job well.
I like to explain my work as "do whatever is needed to do as little work as possible".
Being by improving logs, improving architecture, updating logs, pushing responsibilities around or rejecting some features.
The most clever lines of code are the ones you don’t write. Often this is a matter of properly defining the problem in terms of data structure. LLMs are not at all good at seeing that a data structure is inside out and that by turning it right side in, we can fix half the problems.
More significantly though, OP seems right on to me. The basic functionality of LLMs is handy for a code writing assistant, but does not replace a software engineer, and is not ever likely too no matter how many janky accessories we bolt on. LLMs are fundamentally semantic pattern matching engines, and are only problem solvers in the context of problems that are either explicitly or implicitly defined and solved in their training data. They will always require supervision because there is fundamentally no difference between a useful LLM output and a “hallucination” except the utility rating that a human judge applies to the output.
LLMs are good at solving fully defined, fully solved problems. A lot of work falls into that category, but some does not.
>> The most clever lines of code are the ones you don’t write.
Just to add, I think there are three things that LLMs don't address here, but maybe it's because they're not being asked the broader questions:
1. What are some reasonable out-of-band alternatives to coding the thing I'm being asked to code?
2. What kind of future modifications might the client want, and how can we ensure this mod will accommodate those without creating too many new constraints, but also without over-preparing for something that night not happen?
3. What is the client missing that we're also missing? This could be as simple as forgetting that under some circumstances, the same icon is being used in a UI to mean something else. Or that an error box might obscure the important thing that just triggered the error. Or that six years ago, we created a special user level called "-1" that is a reserved level for employees in training, and users on that level can't write to certain tables. And asking the question whether we want them to be able to train on the new feature, and if so, whether there are exceptions to that which would open the permissions on the DB but restrict some operations in the middleware.
"What are we missing" is 95% of my job, and unit tests are useless if you don't know all the potential valid or invalid inputs.
I jumped off the boat of llm a little before MCP was a thing, so I thought that the tools were presented as needed by the prompt/context in a way not dissimilar of RAG. Isn't this the standard way?
You _can_ build things that way. But then you need some business logic to decide which tools to expose to the system. The easy/dumb way is just to give it all the tools. With RAG, you have retrieval step where you have hardcoded some kind of search (likely semantic) and some kind of pruning or relevance logic (maybe give the top 5 results that have at least X% relevancy matching).
With tools there is no equivalent. Maybe you could try some semantic similarity to the tool description, but I don't know of any system that does that.
What seems to be happening is building distinct "agents" that have a set of tools designed into them. An Agent is a system prompt+tools, where some of tools might be the ability to call/handoff to other agents. Each call to an agent is a new context, albeit with some limited context handed in from the caller agent. That way you are manually decomposing the project into a distinct set of sub-agents that can be concretely reasoned about and can perform a small set of related tasks. Then you need some kind of overall orchestration agent that can handle dispatch to other agents.
I've used to commute in inline skates in traffic. It wasn't usual but I had a couple of close encounters.
The was an avenue with bike preference in the leftmost lane, but as cars parked anyway it way like a 1/4 lane.
Once I was completely zoomed out and felt something was off. The car at my right just push the break.
I followed along with a t drag and straighten up to get a clearer vision. A truck was turning left from the middle of the avenue, cutting 2 lanes including mine.
These things always happens in downhill. I had not enough space to brake. I turned left almost, not without almost hitting a woman that was waiting to cross the street 2 steps down the curb, and headed full blast too a cobblestone street. I don't know how I manage to do that without falling.
Yes a less experienced me would have not seen the risk early enough nor had the skills to get that turn. But there are shit you cannot predict.
In context, they are clearly saying that beginner cyclists should not have to panic stop because it is exceptionally rare, as evidenced by their 14 years without needing to do it. It is extrapolating from their singular, personal experience to general frequency for the population. It is equivalent to saying "people shouldn't worry about breaking bones because I've never broken a bone." Even if you qualify the statement by saying "maybe someone out there has broken a bone at some point" you are still making a statement about it's rarity that can not be adequately supported by your personal anecdote. Having multiple people in a single thread which has been viewed by a few thousand people at most who have counter examples should clearly indicate that "it hasn't happened to me" doesn't equate to "you shouldn't expect it to happen to you."
Maybe I'm more charitable in my interpretation, but I don't really see grandparent making a generalization, they're giving a counter-example to requiring everyone to practice panic stops.
His and my reply assert that panic stops are not a panacea of avoiding traffic problems, but a side-effect of not paying more attention.
And cheers, this is my last clarification. I don't enjoy engaging with such excessive pedantry.
I think battlestar Galactica must be quoting one of the Eddas. I've only read if it from Borges in Spanish, but Conner the same meaning: "Estas cosas han pasado. Estas cosas también pasarán."
