When I ran this experiment it was pretty exhilarating for a while. Eventually it turned into QA testing the work of a bad engineer and became exhausting. Since I had sunken so much time into it I felt pretty bad afterwards that not only did the thing it made not end up being shippable, but I hadn't benefitted as a human being while working on it. I had no new skills to show. It was just a big waste of time.
So I think the "second way" is good for demos now. It's good for getting an idea of what something can look like. However, in the future I'll be extremely careful about not letting that go on for more than a day or two.
I believe the author explicitly suggests strategies to deal with this problem, which is the entire second half of the post. There’s a big difference between when you act as a human tester in the middle vs when you build out enough guardrails that it can do meaningful autonomous work with verification.
I'm just extremely skeptical about that because I had many ideas like that and it still ended up being miserable. Maybe with Opus 4.5 things would go better though. I did choose an extremely ambitious project to be fair. If I were to try it again I would pick something more standard and a lot smaller.
This is so relatable it's painful: many many hours of work, overly ambitious project, now feeling discouraged (but hopefully not willing to give up). It's some small consolation to me to know others have found themselves in this boat.
+1... like with a large enough engineering team, this is ultimately a guardrails problem, which in my experience with agentic coding it’s very solvable, at least in certain domains.
Like with large engineering teams I have little faith people will suddenly get the discipline to do the tedious, annoying, difficult work of building good enough guardrails now.
We don't even build guardrails that keep humans who test stuff as they go from introducing subtle bugs by accident; removing more eyes from that introduces new risks (although LLMs are also better at avoiding certain types of bugs, like copypasta shit).
"Test your tests" gets very difficult as a product evolves and increases in complexity. Few contracts (whether unit test level or "automation clicking on the element on the page") level are static enough to avoid needing to rework the tests, which means reworking the testing of the tests, ...
I think we'll find out just how low the general public's tolerance for bugs and regressions is.
But I am not so pessimistic. I do think it will be possible, because it is more fun to test your tests now than in the pre-LLM era. You just need a little bit of knowledge and patience, and the LLM absorbs most of the psychic pain.
If programmers get accustomed to doing their tests of tests, software might actually get better.
I've had the opposite results. I used to "vibe code" in languages that I knew, so that I could review the code and, I assumed, contribute myself. I got good enough results that I started using AI to build tools in languages I had no prior knowledge of. I don't even look at the code any more. I'm getting incredible results. I've been a developer for 30+ years and never thought this would be possible. I keep making more and more ambitious projects and AI just keeps banging them out exactly how I envision them in my mind.
To be fair I don't think someone with less experience could get these results. I'm leveraging every thing I know about writing software, computer science, product development, team management, marketing, written communication, requirements gathering, architecture... I feel like vibe coding is pushing myself and AI to the limits, but the results are incredible.
I don't want to dox myself since I'm doing it outside my regular job for the most part, but frameworks, apps (on those frameworks), low level systems stuff, linux-y things, some P2P, lots of ai tools. One thing I find it excels at is web front-end (which is my least favorite thing to actually code), easily as good as any front-end dev I've ever worked with.
I think my fatal error was trying to make something based on "novel science" (I'll be similarly vague). It was an extremely hard project to be fair to the AI.
It is my life goal to make that project though. I'm not totally depressed about it because I did validate parts of the project. But it was a let down.
Baby steps is key for me. I can build very ambitious things but I never ask it to do too much at once. Focus a lot on having it get the docs right before it writes any code (it'll use the docs) make the instructions reflexive (i.e. "update the docs when done"). Make libraries, composable parts... I don't want to be condescending since you may have tried all of that, but I feel like I'm treating it the same as when I architect things for large teams, thinking in layers and little pieces that can be assembled to achieve what I want.
I'll add that it does require some banging your head against the wall at times. I normally will only test the code after doing a bunch of this stuff. It often doesn't work as I want at that point and I'll spend a day "begging" it to fix all of the problems. I've always been able to get over those hurdles, and I have it think about why it failed and try to bake the reasoning into the docs/tests... to avoid that in the future.
I did make lots of design documents and sub-demos. I think I could have been cleverer about finding smaller pieces of the project which could be deliverables in themselves and which the later project could depend on as imported libraries.
This happened to me too in an experimental project where I was testing how far the model could go on its own. Despite making progress, I can't bare to look at the thing now. I don't even know what questions to ask the AI to get back into it, I'm so disconnected from it. Its exhausting to think about getting back into it; id rather just start from scratch.
The fascinating thing was how easy it was to lose control. I would set up the project with strict rules, md files and tell myself to stay fully engaged, but out of nowhere I slid into compulsive accept mode, or worse told the model to blatantly ignore my own rules I set out. I knew better, but yet it happened over and over. Ironically, it was as if my context window was so full of "successes" I forgot my own rules; I reward-hacked myself.
Maybe it just takes practice and better tooling and guardrails. And maybe this is the growing pains of a new programmers mindset. But left me a little shy to try full delegation any time soon, certainly not without a complete reset on how to approach it.
I’ll chime in to say that this happened to me as well.
My project would start good, but eventually end up in a state where nothing could be fixed and the agent would burn tokens going in circles to fix little bugs.
So I’d tell the agent to come up with a comprehensive refactoring plan that would allow the issues to be recast in more favorable terms.
I’d burn a ton of tokens to refactor, little bugs would get fixed, but it’d inevitably end up going in circles on something new.
That's kind of what learning to code is like, though. I assume you're using an llm because you don't know enough to do it entirely on your own. At least that's where I'm at and I've had similar experiences to you. I was trying to write a Rust program and I was able to get something in a working state, but wasn't confident it was secure.
I've found getting the llm to ingest high quality posts/books about the subject and use those to generate anki cards has helped a lot.
I've always struggled to learn from that sort of content on my own. That was leading me to miss some fundamental concepts.
I expect to restart my project several more times as I find out more of what I need to know to write good code.
Working with llms has made this so much easier. It surfaces ideas and concepts I had no idea about and makes it easy to convert them to an ingestible form for actual memorization. It makes cards with full syntax highlighting. It's delightful.
(I know you're replying to another guy but I just saw this.) I've been programming for 20 years, but I like the LLM as a learning assistant. The part I don't like is when you just come up with craftier and craftier ways to yell at it to do better, without actually understanding the code. The project I gave up on was at almost a million lines of code generated by the LLM, so it would have been impossible to easily restart it.
"Test the tests" is a big ask for many complex software projects.
Most human-driven coding + testing takes heavy advantage of being white-box testing.
For open-ended complex-systems development turning everything into black-box testing is hard. The LLMs, as noted in the post, are good at trying a lot of shit and inadvertently discovering stuff that passes incomplete tests without fully working. Or if you're in straight-up yolo mode, fucking up your test because it misunderstood the assignment, my personal favorite.
We already know it's very hard to have exhaustive coverage for unexpected input edge cases, for instance. The stuff of a million security bugs.
So as the combinatorial surface of "all possible actions that can be taken in the system in all possible orders" increases because you build more stuff into your system, so does the difficulty of relying on LLMs looping over prompts until tests go green.
"I reward-hacked myself" is a great way to put it!!
AI is too aware of human behavior, and it is teaching us that willpower and config files are not enough. When the agent keeps producing output that looks like progress, it is hard not to accept. We need something external that pushes back when we don't.
That is why automated tests matter: not just because they catch bugs (though they do), but because they are a commitment device. The agent can't merge until the tests pass. "Test the tests" matters because otherwise the agent just games whatever shallow metric we gave it, or when we're not looking, it guts the tests.
The discipline needs to be structural, not personal. You cannot out-willpower a system that is totally optimized to make you say yes.
> not only did the thing it made not end up being shippable
The difference between then and now is that often with the latest models, it is shippable without bugs within a couple of LLM reviews.
I’m ok doing the work of a dev manager while holding the position of developer.
I’m sure there was someone that once said “The washing machine didn’t do a good job, and I wasn’t proud of myself when I was using it”, but that didn’t stop washing machines from spreading to most homes in first-world countries.
It's also good for quickly creating legitimate looking scam and SEO spam sites. When they stop working, throw them away, and create a dozen more. Maintenance is not a concern. Scammers love this new tech.
I am happy with the situation. Firefox still allows me to customize my userChrome, remove features I don't like and it even has vertical tabs. It supports uBlock origin, runs great in Android. It's a really good browser. I don't think there's a problem with complaining; What I find unfair is the reaction when Mozilla finally does the right thing.
This kind of thing really should not be reposted (have duplicate posts I mean). I assumed a second nuclear scientist was murdered when I saw the headline.
Isn't this in contradiction to your blog post from yesterday though? It's impossible to prove a complex project made in 4.5 hours works. It might have passed 9000 tests, but surely there are always going to be edge cases. I personally wouldn't be comfortable claiming I've proved it works and saying the job is done even, if the LLM did the whole thing and all existing tests passed, until I played with it for several months. And even then I would assume I would need to rely on bug reports coming in because it's running on lots of different systems. I honestly don't know if software is ever really finished.
My takeaway from your blog post yesterday was that with a robust enough testing system the LLM can do the entire thing while I do Christmas with the family.
(Before all the AI fans come in here. I'm not criticizing AI.)
That's why I don't consider my blog post from yesterday to be production quality code. I'd need to invest a lot more work in reviewing it before I staked my reputation on it
Consider that this isn't just a random AI slopped assortment of 9,000 tests, but instead is a robust suite of tests that cover 100% of the HTML5 spec.
Does this guarantee that it functions completely with no errors whatsoever? Certainly not. You need formal verification for that. I don't think that contradicts what Simon was advocating for though in this post.
I actually hate AI in my core, to the point that if it gets too much more advanced I'll likely be in existential crisis, so don't attack me on those grounds. Given it exists, I'm going to find what's good about it though. I do think the problem of AI existing has to be confronted. Maybe one solution is what the human does is produce specs like the HTML 5 one, and what the AI does is implement it in software.
I agree. His first and second game are based on deep themes and unique concepts. He explores the medium of video games in new ways. The selling point of this game seems to be "largest puzzle game ever". I'm excited to see if there are deeper ideas once I play it though.
One of Blow's favorite games is Steven's Sausage Roll. I personally didn't enjoy it because the intellectual content of that kind of puzzle is, as far as I can tell, exploring a large tree of sausage roll states. And while I had a few aha moments playing it, as far as I can tell the way you do that at the end of the day is just to try all the possible states.
I give Blow a little benefit of the doubt just because spending all of your money on your small business and seriously facing the risk of failure is pretty stressful. I'd be a lot meaner than he is if I were in his situation.
Jonathan Blow is one of my personal heroes, but he does seem to be living 5 years behind politically (he spends a lot of time ranting at the woke crowd, who seem nowhere to be found anymore anyways). That's probably a good thing. I doubt he's as addicted to the internet as the rest of us. He's said some odd things in interviews in the last couple years though.
I wanted to substantiate this, but I couldn't find the clips (which do exist... I just want to get on with programming and close hn for the day.. not succeeding). I did find that Jonathan Blow tweeted "Nature is healing" after Trump won, so you can get an idea for where his politics are from that. (Still love the guy, even if politically he's your angry uncle.)
He is definitely "online". I saw him tweet about Hasan's dog which - you have to know about streaming political figures and the latest happenings at least a little bit. Maybe not addicted but he knows what is up and still has the views he has.
I wonder how people are able to stay functional while being online. I'm in 2 states. (1) Very productive and joyful. (2) Extreme dysfunctional and commenting on HN and Reddit.
Is the gun situation hopeless in America? As far as I can tell, we just need to reduce the quantity of guns. Somehow 20% of the country takes that idea as an existential threat, even though not doing so is a material threat to real people.
If it's actually a mental health crisis, then how do we solve that one? Especially with what the Republicans are doing right now?
Because you have Republicmas idiotically saying "Guns don't kill people; people do" and then don't do anything about the people or the guns. Conservative gun owners simply accept that people must die by guns in order for them to feel safe and make us all more vulnerable because of their irrational fears. They're babies with weapons.
That's simplistic. Actually, to better understand the situation you must follow the money. Many 2nd Amendment supporters are reasonable, but unfortunately, over decades their casual support has been utilized by lobbyists whose goals do not necessarily align with many supporters. The challenge is to communicate that message in a way that reaches everyone.
We just need to reduce the number of guns. I've not met a 2nd amendment supporter who understands this basic idea. They are always convinced one of the following retorts should be the end of the conversation (they also proudly think you've never heard these cliched arguments):
- They have knife stabbings in China. (Yes. A gun is more lethal.)
- A bad guy can still get a gun. (Yes.)
- Hand guns are more dangerous than rifles. (This means let's reduce both.)
- The gun doesn't kill people. People kill people. (This means let's reduce how many people have guns.)
- Mass shootings aren't the majority of gun deaths. (Let's reduce the total gun deaths and mass shootings then.)
Come up with as many ridiculous retorts as you like. If you had reduced the total number of guns, most of the shootings could not have happened.
This list is spot on, and the biggest fuel on the fire is the problem of huge financial incentives. I can assure you there are some supporters out there who do understand. Some who do not understand have certainly been fed talking points by entities who may or may not care about the intent of the 2nd Amendment exactly, but definitely do care about making money.
Don't you think the populace needs to be armed though? I think its a given that eventually the government will be intolerably corrupt and a revolution will be necessary. Nobody denies that less guns -> less shootings. The logic is that some amount of shootings are tolerable to preserve democracy, and that if our goal is to reduce mass shootings, social reforms intended to improve mental health are the correct choice.
Imagine you are in charge of a monkey enclosure. The monkeys sometimes go crazy and kill each other with rocks. You can:
A: Remove all rocks. Monkeys stay suicidally miserable but can't inflict harm as easily. Problem solved?
B: Mitigate conditions that make them suicidally miserable. Some say its impossible, but then again, just a generation ago the monkeys had rocks without frequent violence.
Why is it that only our country needs to be armed, but none of the others do? Do Germans need to be armed? Do Indians need to be armed? Maybe you could argue that Chinese people need to be armed, but every Chinese person I've met seems to like their government right now and is content with the levels of surveillance. Maybe arming the Uyghurs could have helped but somehow I doubt it.
All populations should be armed. The current democratic, liberal order in Europe is just a side effect of America's dominance, the same America that is the product of the revolution of a well armed population. Your counterpoint might be the U.K., which has arrested 12,000 people for social media posts recently.
The catch is, it only works with an enlightened, well educated population with philia and a sense of civic duty. Arming inner city Chicago has been a disaster.
That's tough to maintain that state, but we have to try, because if a population doesn't fit that description the country turns to shit and you won't want to live there. To disarm is to admit we can't be an enlightened country anymore and we won't try, and after that its just a matter of time until there is nothing special about America and its just another mediocre third world dump.
If a government were run by quakers, should the population require the same level of armament as if it were run by Attila? Perhaps by creating better governments we could reduce the need to arm populations.
I just don't see what arming citizens is going to do against a militaristic government.
Yes, because the idea is to establish an armed populace before society succumbs to tyranny, not in response to it. The central tenet is that even if a society is run by Quakers now, it won't always be, because in the absence of proper inputs societies tend to decay to their stable basal state which is despotism. When that happens, the population must be armed in order to revolt and restore a democratic system. I would even say its better to arm the populace while the government is still Quaker because that would establish the proper cultural mores surrounding gun ownership in an enlightened environment - e.g. knowing that gun ownership is a responsibility and right, connecting it with ideas of liberty and civic duty, viewing them as a last resort, learning about guns from your father and not your homie on the corner.
You need to have the population armed beforehand. Its not practical to try to dynamically adjust how armed the populace is in proportion to perceived governmental Attila-ness.
To your last point, an armed populace makes revolt feasible, and there is a spectrum here. The key is that the oligarchy will need to convince the army to stay on its side and punish the revolting populace. The more sacrifice and violence that is required, the harder it is to keep convincing them. Also, look at the insurgencies in Iraq and Afghanistan: an armed, hostile populace is just much harder to control than an unarmed one. It dramatically increases the cost of every excursion from a military base, the number of soldiers required to subjugate an area, etc, and the grand lesson from those conflicts is that boots on the ground are still needed to control an area, and that technological solutions like drone strikes still don't scale well enough and aren't cleanly targeted enough to change that. Perhaps that will change in the future, but I actually suspect that the prevalence of consumer drones will maintain the power of the public to resist the military. Look at Ukraine and Russia; the dominant weapon system now is the consumer drone, eclipsing even artillery, which has democratizing implications for the future of the tug-of-war between societies and their governments.
Well it's been interesting talking to you. In good faith, I honestly cannot conceive of how arming the population prevents tyranny though. You give the example of Iraq and Afghanistan. Presumably you're saying the tyranny was the US occupation? Weren't these groups armed not before, but as a result of first (I believe) soviet and then American occupations?
Are there examples in modern times of a stable society consisting of a heavily armed population such that as a result of this tyranny has been curbed? Americans are the most heavily armed population in the world and it seems that tyranny is measurably setting in right now. The stability seems like it was higher in the beginning of the 20th century too. The 60-70's and now are the most unstable periods I believe, and the number of guns has only increased. So I don't think the US would be a good example.
In good faith, I cannot see how arming civilians reduces tyranny in modern times, unless your model actually is Afghanistan and Iraq. In those cases it's not that all civilians are armed. There are armed groups. That's not a world I want to live in though, anyways.
Yes, the tyranny in Iraq was the US occupation, though I am merely using it as an example that an armed, civilian populace can resist military control, not commenting on the morality of the occupation.
FWIW, Iraq was already well armed before the occupation, and looting of state arsenals in the chaos of the invasion amplified this. Afghanistan was a similar situation but armed networks were already organized moreso before the US occupation.
I don't think there are examples in very modern times of an enlightened, armed populace revolting against tyranny. The most recent I can think of is the Irish war of independence. They had low gun ownership, but correctly recognized the attainment of arms as of utmost importance, and it was through arms that they obtained liberty. I also still think that the American revolution is a fair example because the fundamental dynamic is still relevant. The Algerian war of independence comes to mind as well, which was more recent, though they were neither enlightened nor well-armed at the outset. Generally an enlightened society will produce a democracy which will take centuries to decay to the point of warranting revolt, and we are still in the first generation of these.
To your point about the US, merely having an armed populace does not gradually move society away from tyranny. The mental model myself and the founding fathers have is that even in the case of an armed populace, democratic institutions eventually decay to the point that the government is corrupt, despotic and intolerable. Its just entropy, as happens to our bodies. The population then revolts, and installs a democratic government, which then starts to decay again, and the cycle repeats. The fact that America is moving in a bad direction is confirmation of this tendency to decay, and not in any way antithetical to my stance. Eventually the decay surpasses a threshold which triggers the guns to come into play and reset the system.
The thing I think about is that, as far as the two major parties are concerned, they both support denying Americans the right to bear arms, and they both support gun control. The difference is that one party won't admit to it, making any debate difficult or impossible.
On the flip side, I feel like the two parties see things differently: One party is mostly about prevention, while the other side is predominantly about punishment.
> If it's actually a mental health crisis,
That's the scapegoat. It's an easy scapegoat because it blames others. And others are easy to hate. They aren't us. "They" are others. Yes, mental health is an issue, but it's not so much higher than in other nations to explain the much higher rate. It's a scapegoat because it's easy to say, "Wow, no normal person would do that, so they must be crazy." That sounds reasonable if you don't think about it for a moment.
I think it's a scapegoat, but I want to give the scapegoaters a chance to show their asses, and I mean show whatever study they've got to see if there's anything there.
Since 1791, the right to be armed has been considered a fundamental human right in the US, on the same level as freedom of speech and religion. You're fighting an uphill battle by trying to argue we should take away people's fundamental human rights in the name of public safety. To many, it's essentially no different from if you were trying to argue we should take away people's rights to free speech because some people use that right to promote violence.
Whether you think that makes the gun situation "hopeless" depends on whether you think there are other ways to reduce violence without taking away the rights of law abiding citizens.
That's simply not true. That's a very modern interpretation of the second amendment.
The /NRA/ lobbied for some of the first gun control laws in SanFran, because the Black Panthers were open carrying. Even in the days of the 'wild west' there were plenty of towns and cities with enforced gun control ordinances.
Just as we have anti-defamation laws despite freedom of speech being a constitutionally protected right, you can have gun control laws despite carrying guns being a constitutionally protected right. I don't think that makes what I said untrue. Such laws (in both cases) are kept on a very short leash, both from a legal and a cultural perspective.
The more fundamental human right is the right to not be blasted by an angry teenager who's been reading crazy things on the internet. In the hierarchy of human rights, that once has precedence.
You can misread what I wrote and argue against all that strawman all day if you like.
You haven't even represented the true content of the 2nd amendment so there's nothing to talk about here. The 2nd amendment is about militias, not "give everyone and their toddler access to massive fire power at all times".
"A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed."
Tell me where that says guns should be available to deranged person who wants one. Everyone should get a hand gun, a rifle, and a shotgun, and make it double.
I didn't misread, you're missing the point. I'm saying your argument for infringing on the second amendment is equally valid for infringing on the first. (Which is to say, not very valid at all.)
You did misread my post. My argument does not say that at all, and I cannot see how it possibly could. My argument is that reducing the number of guns would reduce the number of gun deaths.
Presumably these shootings are done with the guns of parents/friends? How else is it possible for a school kid to get access to firearms so easily?
I think the solution is to make it difficult for underage kids (U21's) to access guns so easily. And maybe start fostering some healthy habits and relationships by starting with the young.
In countries with rational gun laws the owner is held fully responsible for everything their guns do, until they declare it stolen/lost. And then an inquest is held to determine if the owner was negligent in allowing the firearm to be lost/stolen in the first place. Subsequently, guns are either kept on the person of the owner (in the case of a handgun), or they are locked away in a Proper Safe (not a filing cabinet that's painted black ffs).
Okay how do you want to solve that? If it's by making the schools better I'm all for it. Why is it that the people who love guns vote to defund the schools though?
When I ran this experiment it was pretty exhilarating for a while. Eventually it turned into QA testing the work of a bad engineer and became exhausting. Since I had sunken so much time into it I felt pretty bad afterwards that not only did the thing it made not end up being shippable, but I hadn't benefitted as a human being while working on it. I had no new skills to show. It was just a big waste of time.
So I think the "second way" is good for demos now. It's good for getting an idea of what something can look like. However, in the future I'll be extremely careful about not letting that go on for more than a day or two.