I'm a huge admirer of Jonathan Blow but I'm not sure his perspective is very reflective of reality here. He's a notoriously picky engineer and also tends to work on very hard problems and working at a very low level. e.g. writing his own language (Jai) and compiler because he thinks existing languages (C++ and Rust) are not good enough for game development.
So sure, I am not surprised that no current AI/LLM is good enough for his standard, but I'm sure 99.99% of engineers on Hackernews are not good enough for his standard either. Again, huge admirer of his work and talks, he's clearly very gifted but he seems like one of those engineers who probably thinks his co-workers are all morons.
I've worked at a variety of startups, enterprise companies, game companies, some you've heard of and some you haven't, and I've seen the whole gamut of code and engineers. I've seen basic shitty CRUD apps which power customer service teams of 1000s and I've seen physics and low-level audio programming which only a handful of dudes in the company could understand.
There is a massive variety of systems and code out there in the wild and I think Jonathan Blow would be surprised at the amount of revenue generated by extremely shitty code written by humans.
From my perspective, the current batch of LLMs and tools are MASSIVELY capable of writing production code, in fact, far, far better than so many real production systems I've seen. This can be true while it's also true that I wouldn't have any current LLM write me a low level game engine in C or work on the Linux kernel, or work on low level database code etc.
People generalizing about how bad LLMs are at coding are often doing it out of a very unusual perspective (like J Blow) or doing it out of bad faith. Or perhaps haven't really seen the reality that there's a LOT of code out there which is terrible and still does the job and still keeps people employed.
> Or perhaps haven't really seen the reality that there's a LOT of code out there which is terrible and still does the job and still keeps people employed.
1. There is a big difference between terrible code that works in production, and terrible code that doesn't even work. And tge vast majority of the code I've generated by my LLM doesn't even work.
2. Software is inherently useful to enable change. If you don't want to change it, burn the program onto a chip and have it run faster. But we are here talking because you want to be able to change it. Terrible code makes change harder, riskier, slower, more expensive. The curves have already been discovered by others, I am not inventing any new knowledge here: there are software projects where longevity is of buisness criticality and there are others where the project will be thrown away in 1,2,3,5 years. If you'll throw it away, sure go ahead and vibe it. If you want it to stay, you're gonna have a bad time as changes become more and more expensive.
> People generalizing about how bad LLMs are at coding are often doing it out of a very unusual perspective (like J Blow) or doing it out of bad faith.
Thats why the only study up to date on developer productivity with AI shows 19% reduction of productivity and the most up to date study on LLM usage in the enterprise show that 95% of projects with LLMs fail.
I'd say its bad faith to claim LLMs are useful considering all studies so far show the opposite.
So sure, I am not surprised that no current AI/LLM is good enough for his standard, but I'm sure 99.99% of engineers on Hackernews are not good enough for his standard either. Again, huge admirer of his work and talks, he's clearly very gifted but he seems like one of those engineers who probably thinks his co-workers are all morons.
I've worked at a variety of startups, enterprise companies, game companies, some you've heard of and some you haven't, and I've seen the whole gamut of code and engineers. I've seen basic shitty CRUD apps which power customer service teams of 1000s and I've seen physics and low-level audio programming which only a handful of dudes in the company could understand.
There is a massive variety of systems and code out there in the wild and I think Jonathan Blow would be surprised at the amount of revenue generated by extremely shitty code written by humans.
From my perspective, the current batch of LLMs and tools are MASSIVELY capable of writing production code, in fact, far, far better than so many real production systems I've seen. This can be true while it's also true that I wouldn't have any current LLM write me a low level game engine in C or work on the Linux kernel, or work on low level database code etc.
People generalizing about how bad LLMs are at coding are often doing it out of a very unusual perspective (like J Blow) or doing it out of bad faith. Or perhaps haven't really seen the reality that there's a LOT of code out there which is terrible and still does the job and still keeps people employed.