This paper argues that if superintelligence can give everyone the health of a 20 year-old, we should accept a 97% percent chance of superintelligence killing everyone in exchange for the 3% chance the average human lifespan rises to 1400 years old.
There is no "should" in the relevant section. It's making a mathematical model of the risks and benefits.
> Now consider a choice between never launching superintelligence or launching it immediately, where the latter carries an % risk of immediate universal death. Developing superintelligence increases our life expectancy if and only if:
> [equation I can't seem to copy]
> In other words, under these conservative assumptions, developing superintelligence increases our remaining life expectancy provided that the probability of AI-induced annihilation is below 97%.
Mr. Superintelligence increasing Nick Bostrom's life expectancy to a trillion years but killing everyone else would as well. Why is he showing us the grace of letting us tag along with him into The Singularity in this fantasy just because we happened to be alive at the same time? Is it because he needs someone to do the actual work?
That's what the paper says. Whether you would take that deal depends on your level of risk aversion (which the paper gets into later). As a wise man once said, death is so final. If we lose the game we don't get to play again.
Everyone dies. And if your lifespan is 1400 years, you won't live for nearly 1400 years. OTOH, people with a 1400 year life expectancy are likely to be extremely risk averse in re anything that could conceivably threaten their lives ... and this would have consequences in re blackmail, kidnapping, muggings, capital punishment, and other societal matters.
- Peter has spent the last year building up a large assortment of CLIs to integrate with. He‘s also a VERY good iOS and macOS engineer so he single handedly gave clawd capabilities like controlling macOS and writing iMessages.
- Leaning heavily on the SOUL.md makes the agents way funnier to interact with. Early clawdbot had me laugh to tears a couple times, with its self-deprecating humor and threatening to play Nickelback on Peter‘s sound system.
- Molt is using pi under the hood, which is superior to using CC SDK
- Peter’s ability to multitask surpasses anything I‘ve ever seen (I know him personally), and he’s also super well connected.
Check out pi BTW, it’s my daily driver and is now capable to write its own extensions. I wrote a git branch stack visualizer _for_ pi, _in_ pi in like 5 minutes. It’s uncanny.
I've been really curious about pi and have been following it but haven't seen a reason to switch yet outside anecdotes. What makes it a better daily driver out of the box compared to Claude or Codex? What did you end up needing to add to get your workflow to be "now capable to write its own extensions"? Just trying to see what the benefit would be if I hop into a new tool.
Why don’t you try it, it’s 2 minutes to setup (or tell Claude to do it), and it uses your CC Max sub if you want.
Some advantages:
- Faster because it does no extra Haiku inference for every prompt (Anthropic does this for safety it seems)
- Extensions & skills can be hot reloaded. Pi is aware of its own docs so you just tell it „build an extension that does this and that“. Things like sub agents or chains of sub agents are easily doable. You could probably make a Ralph workflow extension in a few minutes if you think that’s a good idea.
- Tree based history rewind (no code rewind but you could make an extension for that easily)
- Readable session format (jsonl) - you can actually DO things with your session files like analysis or submit it along with a PR. People have workflows around this already. Armin Ronacher liked asking pi about other user’s sessions to judge quality.
- No flicker because Mario knows his TUI stuff. He sometimes tells the CC engs on X how they could fix their flicker but they don’t seem to listen. The TUI is published separately as well (pi-tui) and I‘ve been implementing a tailing log reader based on it - works well.
Sure, I'm not using it with my company/enterprise account for that reason. But for my private sub, it's worth the tradeoff/risk. Ethically I see no issue at all, because those LLMs are trained on who knows what.
But you can use pi with z.ai or any of the other cheap Claude-distilled providers for a couple bucks per month. Just calculate the risk that your data might be sold I guess?
It’s vibe coded slop that could be made by anyone with Claude Code and a spare weekend.
It didn’t require any skill, it’s all written by Claude. I’m not sure why you’re trying to hype up this guy, if he didn’t have Claude he couldn’t have made this, just like non engineers all over the world are coding all a variety of shit right now.
I’ve been following Peter and his projects 7-8 months now and you fundamentally mischaracterize him.
Peter was a successful developer prior to this and an incredibly nice guy to boot, so I feel the need to defend him from anonymous hate like this.
What is particularly impressive about Peter is his throughput of publishing *usable utility software*. Over the last year he’s released a couple dozen projects, many of which have seen moderate adoption.
I don’t use the bot, but I do use several of his tools and have also contributed to them.
There is a place in this world for both serious, well-crafted software as well as lower-stakes slop. You don’t have to love the slop, but you would do well to understand that there are people optimizing these pipelines and they will continue to get better.
Weekend - certainly not, the scope is massive. All those CLIs - gmail, whisper, elevenlabs, whatsapp/telegram/discord/etc, obsidian, generic skills marketplace etc, it's just so many separate APIs to build against.
But Peter just said in his TBPN interview that you can likely re-build all that in 1 month. Maybe you'd need to work 14h per day like he does, and running 10 codex sessions in parallel, using 4-6 OpenAI Pro subs.
hard to do "credit assignment", i think network effects go brrrrrr. karpathy tweeted about it, david sacks picked it up, macstories wrote it up. suddenly ppl were posting screenshots of their macmini setups on x and ppl got major FOMO watching their feeds. also peter steinberger tweets a lot and is prolific otherwise in terms posting about agentic coding (since he does it a lot)
its basically claude with hands, and self-hosting/open source are both a combo a lot of techies like. it also has a ton of integrations.
will it be important in 6 months? i dunno. i tried it briefly, but it burns tokens like a mofo so I turned it off. im also worried about security implications.
It's totally possible Peter was the right person to build this project – he's certainly connected enough.
My best guess is that it feels more like a Companion than a personal agent. This seems supported by the fact I've seen people refer to their agents by first name, in contexts where it's kind of weird to do.
But now that the flywheel is spinning, it can clearly do a lot more than just chat over Discord.
Yeah makes sense. Something about giving an agent its own physical computer and being able to text it instructions like a personal assistant just clicks more than “run an agent in a sandbox”.
Since you have not mentioned it: those crypto scams are NOT related to the project in _any_ way. And I really doubt they've helped the popularity much. From the creator himself: https://x.com/steipete/status/2016072109601001611
It's not. The guy behind Moltbot dislikes crypto bros as much as you seem to. He's repeatedly publicly refused to take fees for the coin some unconnected scumbags made to ride the hype wave, and now they're attacking him for that and because he had to change the name. The Discord and Peter's X are swamped by crypto scumbags insulting him and begging him to give his blessing to the coin. Perhaps you should do a bit of research before mouthing off.
i'd say the crypto angle is only one factor. as is usual in the real world, effects are multifactorial.
clawdbot also rode the wave of claude-code being popular (perhaps due to underlying models getting better making agents more useful). a lot of "personal agents" were made in 2024 and early 2025 which seem to be before the underlying models/ecosystems were as mature.
no doubt we're still very early in this wave. i'm sure google and apple will release their offerings. they are the 800lb gorillas in all this.
I stumbled on Brain of the Firm by Stafford Beer as a freshman in college, and loved the idea of an auto-optimizing business.
Back then I had questions about exactly how such a system could be implemented, the algorithm was very hand wavey, but I assumed surely they must’ve figured it out before writing a book about it.
As an adult with 20 years extra experience, I’m fairly confident that, no, aside from the high level concept, they had no idea how to build such a system. That coup was probably the best possible outcome for Beer - it gave credibility to his ideas without actually testing them.
There is an argument to be made that that companies like Walmart and Amazon operate as planned economies. They use the same cybernetic principles, real time data monitoring and feedback loops, to solve logistics and planning. These implementations do give credibility Beer's ideas.
There is even a section about this in the wiki article:
Any larger corporation does this, since at least decades.
It's called "strategische Konzernentwicklung" in german, meaning "strategic development(forecasting) of the corporation" and its markets. A global insurance company I've worked for had something like this https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ... in 2001. I've been involved in planning, installing and operating it. But not the responsible patsy :-) . Which didn't went that smooth, because most users were higher management, and needed holding hands for all the 'complicated stuff', like loading in data, and playing scenarios with those. Also bulky 3D-glasses, and jerky updates, making most people dizzy when standing. All in all several million of Euros for fancy Silicon Graphics hardware and supercustom wall displays and projection, with not so fancy OS and applications. Excel was percieved as more 'productive'.
The demo Formula 1 simulator had its fans, though :-)
Doing this within one organization, with modern technology, is clearly possible. Attempting this across an economy, in the 70's, where a key premise is "assume you have clean realtime data across all industries," is a fool's errand :) That the ideas sound similar is like arguing Stockfish is based on the original Mechanical Turk. Only true in a superficial sense.
> the difference between high and low load programming within [newbies]
Fixed that.
As the comment you replied to noted, newbie gains are remarkably sensitive to any stimulation, and insensitive to the type of stimulation. Because going from zero to any resistance training is a massive stimulus increase, on a long-term under stimulated system.
The study does confirm that. The data it produces is useful.
What this study doesn't do, is help newbies (or anyone) choose the most effective practices to adopt. Because 10 weeks is way too short to identify best practices for any sustained program.
that's fair but the post was on page 3 for a while. glad to see it restored to the front page. (the charitable explanation is that non-moderators can flag stories, as opposed to an official policy to protect YC companies)
reply