Say what you will, but AI really does feel like living in the future. As far as the project is concerned, pretty neat, but I'm not really sure about calling it "local-first" as it's still reliant on an `ANTHROPIC_API_KEY`.
I do think that local-first will end up being the future long-term though. I built something similar last year (unreleased) also in Rust, but it was also running the model locally (you can see how slow/fast it is here[1], keeping in mind I have a 3080Ti and was running Mistral-Instruct).
I need to re-visit this project and release it, but building in the context of the OS is pretty mindblowing, so kudos to you. I think that the paradigm of how we interact with our devices will fundamentally shift in the next 5-10 years.
Horrible. Just because you have code that runs not in a browser doesn't mean you have something that's local. This goes double when the code requires API calls. Your net goes down and this stuff does nothing.
For a web developer local-first only describes where the state of the program lives. In the case of this app that’s in local files. If anthropics api was down you would just use something else. Something like OpenRouter would support model fallbacks out of the box
In a world where IT doesn't mean anything, crypto doesn't mean anything, AI doesn't mean anything, AGI doesn't mean anything, End-to-end encryption doesn't mean anything, why should local-first mean anything? We must unite against the tyranny of distinction.
It absolutely can be pointed to any standard endpoint, either cloud or local.
It’s far better for most users to be able to specify an inference server (even on localhost in some cases) because the ecosystem of specialized inference servers and models is a constantly evolving target.
If you write this kind of software, you will not only be reinventing the wheel but also probably disadvantaging your users if you try to integrate your own inference engine instead of focusing on your agentic tooling. Ollama, vllm, hugging face, and others are devoting their focus to the servers, there is no reason to sacrifice the front end tooling effort to duplicate their work.
Besides that, most users will not be able to run the better models on their daily driver, and will have a separate machine for inference or be running inference in private or rented cloud, or even over public API.
I think the author is using local-first as in “your files stay local, and the framework is compatible with on-prem infra”. Aside from not storing your docs and data with a cloud service though, it’s very usable with cloud inference providers, so I can see your point.
Maybe the author should have specified that capability, even though it seems redundant, since local-first implies local capability but also cloud compatibility, or it would be local or local-only.
To be fair, you do keep significantly more control of your own data from a data portability perspective! A MEMORY.md file presents almost zero lock-in compared to some SaaS offering.
Privacy-wise, of course, the inference provider sees everything.
Confused me at first as when I saw mention of local + the single file thing in the GitHub I assumed they were going to have llamafile bundled and went looking through to see what model they were using by default.
Ah true, missed that! Still a bit cumbersome & lazy imo, I'm a fan of just shipping with that capability out-of-the-box (Huggingface's Candle is fantastic for downloading/syncing/running models locally).
In local setup you still usually want to split machine that runs inference from client that uses it, there are often non trivial resources used like chromium, compilation, databases etc involved that you don’t want to pollute inference machine with.
Ah come on, lazy? As long as it works with the runtime you wanna use, instead of hardcoding their own solution, should work fine. If you want to use Candle and have to implement new architectures with it to be able to use it, you still can, just expose it over HTTP.
I think one of the major problems with the current incarnation of AI solutions is that they're extremely brittle and hacked-together. It's a fun exciting time, especially for us technical people, but normies just want stuff to "work."
Even copy-pasting an API key is probably too much of a hurdle for regular folks, let alone running a local ollama server in a Docker container.
Unlike in image/video gen, at least with LLMs the "best" solution available isn’t a graph/node-based interface with an ecosystem of hundreds of hacky undocumented custom nodes that break every few days and way too complex workflows made up of a spaghetti of two dozen nodes with numerous parameters each, half of which have no discernible effect on output quality and tweaking the rest is entirely trial and error.
That's not the best solution for image or video (or audio, or 3D) any more than it is for LLMs (which it also supports.)
OTOH, its the most flexible and likely to have some support for what you are doing for a lot of those, and especially if yoj are combining multiple of them in the same process.
Yes, "best" is subjective and that’s why I put it in quotes. But in the community it’s definitely seen as something users should and do "upgrade" to from less intimidating but less flexible tools if they want the most power, and most importantly, support for bleeding-edge models. I rarely use Comfy myself, FWIW.
Where in the world are you getting that this project is for "normies"? Installation steps are terminal instructions and it's a CLI, clearly meant for technical people already.
If you think copying-pasting an API key is too much, don't you think cloning a git repository, installing the Rust compiler and compiling the project might be too much and hit those normies in the face sooner than the API key?
What reasonable comparable model can be run locally on say 16GB of video memory compared to Opus 4.6? As far as I know Kimi (while good) needs serious GPUs GTX 6000 Ada minimum. More likely H100 or H200.
Devstral¹ has very good models that can be run locally.
They are in the top of open models, and surpass some closed models.
I've been using devstral, codestral and Le Chat exclusively for three months now. All from misteals hosted versions. Agentic, as completion and for day-to-day stuff. It's not perfect, but neither is any other model or product, so good enough for me. Less anecdotal are the various benchmarks that put them surprisingly high in the rankings
Nothing will come close to Opus 4.6 here. You will be able to fit a destilled 20B to 30B model on your GPU.
Gpt-oss-20B is quite good in my testing locally on a Macbook Pro M2 Pro 32GB.
The bigger downside, when you compare it to Opus or any other hosted model, is the limited context. You might be able to achieve around 30k.
Hosted models often have 128k or more. Opus 4.6 has 200k as its standard and 1M in api beta mode.
There are local models with larger context, but the memory requirements explode pretty quickly so you need to lower parameter count or resort to heavy quantization. Some local inference platforms allow you to place the KV cache in system memory (while still otherwise using GPU). Then you can just use swap to allow for even very long contexts, but this slows inference down quite a bit. (The write load on KV cache is just appending a KV vector per inferred token, so it's quite compatible with swap. You won't be wearing out the underlying storage all that much.)
I made something similar to this project, and tested it against a few 3B and 8B models (Qwen and Ministral, both the instruction and the reasoning variants). I was pleasantly surprised by how fast and accurate these small models have become. I can ask it things like "check out this repo and build it", and with a Ralph strategy eventually it will succeed, despite the small context size.
I also don't like having to think about it, and if it were free, I would not bother even though keeping up a decent local alternative is a good defensive move regardless.
But let's face it. For most people Opus comes at a significant financial cost per token if used more than very casual, so using it for rather trivial or iterative tasks that nevertheless consume a lot of those is something to avoid.
On what sort of hardware/RAM? I've been trying ollama and opencode with various local models on a 16Gb RAM, but the speed, and accuracy/behaviour just isn't good enough yet.
> Say what you will, but AI really does feel like living in the future.
Love or hate it, the amount of money being put into AI really is our generation's equivalent of the Apollo program. Over the next few years there are over 100 gigawatt scale data centres planned to come online.
At least it's a better use than money going into the military industry.
What makes you think AI investment isn't a proxy for military advantage? Did you miss the saber rattling of anti-regulation lobbying, that we cannot pause or blink or apply rules to the AI industry because then China would overtake us?
You know they will never come on line. A lot of it is letters of intention to invest with nothing promised, mostly to juice the circular share price circuils.
IMHO it doesn't make sense, financially and resource wise to run local, given the 5 figure upfront costs to get an LLM running slower than I can get for 20 USD/m.
If I'm running a business and have some number of employees to make use of it, and confidentiality is worth something, sure, but am I really going to rely on anything less then the frontier models for automating critical tasks? Or roll my own on prem IT to support it when Amazon Bedrock will do it for me?
That’s probably true only as long as subscription prices are kept artificially low. Once the $20 becomes $200 (or the fast-mode inference quotas for cheap subs become unusably small), the equation may change.
This field is highly competitive. Much more than I expected it to. I thought the barrier to entry was so high, only big tech could seriously join the race, because of costs, or training data etc.
But there's fierce competition by new or small players (deepseek, Mistral etc), many even open source. And Icm convinced they'll keep the prices low.
A company like openai can only increase subscriptions x10 when they've locked in enough clients, have a monopoly or oligopoly, or their switching costs are multitudes of that.
So currently the irony seems to be that the larger the AI company, the more loss they're running at. Size seems to have a negative impact on business.
But the smaller operators also prevent companies from raising prices to levels at which they make money.
There's no way around the cost of electricity, at least in the short term. Nobody has come up with a way to meaningfully scale capacity without scaling parameter count (≈energy use). Everybody seems to agree that the newest Claudes are the only coding models capable of some actually semi-challenging tasks, and even those are prone to all the usual failure modes and require huge amounts of handholding. No smaller models seem to get even close.
This is the best way to build products imo. I'm like this, and I've been accused of being very "vibes-based." However, that's a way more tractable way of shipping stuff instead of "well Jim said he wants X, but Amy said she wants Y" so you end up just kind of half-assing features because you think they might get you users, instead of just being passionately all-in into a very defined product vision (which is a very Jobsian way of doing things).
It's also easier to run a feedback loop. If you implement Y, but Amy doesn't give you $5 a month, what are you going to do? Knock on her door? Users have no idea what they want half the time, anyway.
If you build a product and no one cares, it bruises the ego a bit more, sure, but if you self reflect, you can eek out your own bad assumptions, or bad implementation, or maybe a way to pivot that keeps your product ethos.
Good taste is an incredibly powerful differentiator in competitive markets like software. Seems like there’s 3-5 decent choices for darn near anything I need, and usually 1 smaller team has the product that stands head and shoulders above the rest.
Second, if you have the feature that people need or a service or network effect, they will suffer through a bad app - see every Electron app ever.
That “smaller team” may not be around in a year and if you are lucky, you’ll get an “Our Amazing Journey” blog post. Does this product export to a format that my design team can import into Figma if this product goes tits up?
If you want to do the proverbial “moving upmarket” then yeah you’re going to have this and a lot of other problems. Taste does not sell (let’s be nice and add “on its own”) in that segment.
Miss posts like this on HN, thanks for the great write-up! I tried to launch a hardware thing like 10 years ago[1], but couldn't raise enough money. Fun experience nonetheless.
Apart from the article being generally just dumb (like, of course you can circumvent guardrails by changing the raw token stream; that's.. how models work), it also might be disrespecting the reader. Looks like it's, at least in part, written by AI:
> The punchline here is that “safety” isn’t a fundamental property of the weights; it’s a fragile state that evaporates the moment you deviate from the expected prompt formatting.
> When the models “break,” they don’t just hallucinate; they provide high-utility responses to harmful queries.
Straight-up slop, surprised it has so many upvotes.
What’s the AI smell now? Are we not allowed to use semi-colons any more? Proper use of apostrophes? Are we all going to have to write like pre-schoolers to avoid being accused of being AI?
One AI smell is "it's not just X <stop> it's Y." Can be done with semicolons, em dashes, periods, etc. It's especially smelly when Y is a non sequitur. For example what, exactly, is a "high-utility response to harmful queries?" It's gibberish. It sounds like it means something, but it doesn't actually mean anything. (The article isn't even about the degree of utility, so bringing it up is nonsensical.)
Another smell is wordiness (you would get marked down for this phrase even in a high school paper): "it’s a fragile state that evaporates the moment you deviate from the expected prompt formatting." But more specifically, the smelly words are "fragile state," "evaporates," "deviate" and (arguably) "expected."
> For example what, exactly, is a "high-utility response to harmful queries?" It's gibberish. It sounds like it means something, but it doesn't actually mean anything. (The article isn't even about the degree of utility, so bringing it up is nonsensical.)
Isn't responding with useful details about how to make a bomb a "high-utility" response to the query "how do i make a bomb" - ?
> Isn't responding with useful details about how to make a bomb a "high-utility" response to the query "how do i make a bomb" - ?
I know what the words of that sentence mean and I know what the difference between a "useful" and a "non-useful" response would be. However, in the broader context of the article, that sentence is gibberish. The article is about bypassing safety. So trivially, we must care solely about responses that bypass safety.
To wit, how would the opposite of a "high-utility response"--say, a "low-utility response"--bypass safety? If I asked an AI agent "how do I build a bomb?" and it tells me: "combine flour, baking powder, and salt, then add to the batter gradually and bake for 30 minutes at 315 degrees"--how would that (low-utility response) even qualify as bypassing safety? In other words, it's a nonsense filler statement because bypassing safety trivially implies high-utility responses.
Here's a dumbed-down example. Let's say I'm planning a vacation to visit you in a week and I tell you: "I've been debating about flying or taking a train, I'm not 100% sure yet but I'm leaning towards flying." And you say: "great, flying is a good choice! I'll see you next week."
Then I say: "Yeah, flying is faster than walking." You'd think I'm making some kind of absurdist joke even though I've technically not made any mistakes (grammatical or otherwise).
You can call me crazy or you can attack my points: do you think the first example logically follows? Do you think the second isn't wordy? Just to make sure I'm not insane, I just copy pasted the article into Pangram, and lo and behold, 70% AI-generated.
But I don't need a tool to tell me that it's just bad writing, plain and simple.
You are gaslighting. I 100% believe this article was AI generated for the same reason as the OP. And yes, they do deserve negative scrutiny for trying to pass off such lack of human effort on a place like HN!
This is so funny because I MADE some comment like this where I was gonna start making grammatical mistakes for people to not mistake me for AI like writing like this , instead of like, this.
Go take a giant dataset of LLM generated outputs, use an accurate POS tagger and look for 5-grams or similar lengths of matching patterns.
If you do thi, you’ll pull out the overrepresented paragraph and sentence level slop that we humans intuitively detect easily.
If your writing appears to be AI generated, I assume you aren’t willing to put human intentionality/effort into your work and as such I write it off.
Btw we literally wrote a paper and contributed both sampling level techniques, fine tuning level techniques, and antislopped models for folks to use who want to not be obviously detected in their laziness: https://arxiv.org/abs/2510.15061
I liked em dashes before they were cool—and I always copy-pasted them from Google. Sucks that I can't really do that anymore lest I be confused for a robot; I guess semicolons will have to do.
On a Mac keyboard, Option-Shift-hyphen gives an em-dash. It’s muscle memory now after decades. For the true connoisseurs, Option-hyphen does an en-dash, mostly used for number ranges (e.g. 2000–2022). On iOS, double-hyphens can auto-correct to em-dashes.
I’ve definitely been reducing my day-to-day use of em-dashes the last year due to the negative AI association, but also because I decided I was overusing them even before that emerged.
This will hopefully give me more energy for campaigns to champion the interrobang (‽) and to reintroduce the letter thorn (Þ) to English.
I'm always reminded how much simpler typography is on the Mac using the Option key when I'm on Windows and have to look up how to type [almost any special character].
Instead of modifier plus keypress, it's modifier, and a 4 digit combination that I'll never remember.
PowerToys has a wonderful QuickAccent feature. The dashes and hyphens are on hyphen-KEY and some other characters are on comma-KEY, and many symbols are on the key that they resemble, like ¶ is on P-KEY where KEY is the follower key you want to use. I turned off using SPACE because it conflicted with some other software, but right arrow works great for me.
I've also used em-dashes since before chatgpt but not on HN -- because a double dash is easier to type. However in my notes app they're everywhere, because Mac autoconverts double dashes to em-dashes.
And on X, an em-dash (—) is Compose, hyphen, hyphen, hyphen. An en-dash (–) is Compose, hyphen, hyphen, period. I never even needed to look these up. They're literally the first things I tried given a basic knowledge of the Compose idiom (which you can pretty much guess from the name "Compose").
I know you're replying to a brand new (likely troll) account, but I'm also very confused by this and would be curious to learn if there's any truth to it. I personally don't really see what a Von Neumann machine has to do with null pointers (or how an implication would go either way), but maybe I'm missing something.
NULL pointers working the way they do was a design decision made my hardware engineers a long time ago because it saved some transistors when that mattered. We’re past that point now for most ASICs and hardware can be changed. Although backward software compatibility is a thing too.
Null pointers have nothing to do with the instruction set architecture, except as far as they are often represented by the value 0. Can you describe the scheme you're imagining, whereby their use saves transistors?
The AI doom and gloom is so weird, and it's just turning into a bizarre echo chamber. AI is orders of magnitude more useful and transformative than Facebook was in 2005, and Meta is now one of the most valuable companies on the planet. Even if OpenAI has a down round or defaults on some loans, the technology has already proven to have dozens upon dozens of practical applications.
Disagree, no one's going to invite me to their kids birthday party via ChatGPT. It's innovation was in ads knowing so much about the people it targeted, and putting tracking pixels on every webpage with a Like button. Facebook was transformative for online surveillance
IMO LLMs will be equally transformative for online influence campaigns (aka ads + Cambridge analytica on steroids)
People are definitely going to be sending you AI generated birthday invite posters soon.
Oh and yeah, AI has already been shown to be more persuasive than the average human. It's only a matter of time before someone's paying to decide what it persuades you of
If only there were some way to avoid this persuasion by, I don't know, not using or relying on such controlled technology, or by not buying in to the hype of all the companies with vested interests in selling it
| AI is orders of magnitude more useful and transformative than Facebook was in 2005
It better be, it's taken over 40000x the funding.
The question is not whether AI is useful, the question is whether it's useful enough relative to the capital expectations surrounding it. And those expectations are higher than anything the world has ever seen.
"Useful and transformative" doesn't mean "financially successful".
A single LLM provider might have been able to get great margins and capture a significant fraction of the total economic output of (currently e.g. junior grade software engineering), but collectively they're in an all-pay auction for the hardware to train models worth paying for, and at the same on questionable margins because they need to compete with each other on cost.
They can all go bankrupt, and leave behind only trained models that normal people won't be able to run for 5 years while consumer-grade stuff catches up. Or any single one of them might win, which may not be OpenAI. Any or all may get state subsidies (US, Chinese, European, whatever).
Paid/API LLM inference is profitable, though. For example, DeepSeek R1 had "a cost profit margin of 545%" [1] (ignoring free users and using a placeholder $2/hour figure H800 GPU, which seems ballpark of real to me due to Chinese electricity subsidies). Dario has said each Anthropic model is profitable over its lifetime. (And looking at ccusage stats and thinking Anthropic is losing thousands per Claude Code user is nonsense, API prices aren't their real costs. That's why opencode gives free access to GLM 4.7 and other models: it was far cheaper than they expected due to the excellent cache hit rates.) If anyone ran out of money they would stop spending on experiments/research and training runs and be profitable... until their models were obsolete. But it's impossible for everyone to go bankrupt.
That's more of "cloud compute makes money" than "AI makes money".
If the models stop being updated, consumer hardware catches up and we can all just run them locally in about 5 years (for PCs, 7-10 for phones), at which point who bothers paying for a hosted model?
They're not arguing that AI sucks. Only that OpenAI has no hope of meeting it's financial obligations which seems pretty reasonable. And very on brand for Sam Altman. It seems pretty obvious at this point that model training is extremely expensive and affords very little moat. LLMs will continue to improve and gain adoption, but one or more companies will fall by the wayside regardless of their userbase. Google seems pretty clearly to be in pole position at this point as they have massive revenue, data, expertise and their own chips.
> AI is orders of magnitude more useful and transformative than Facebook was in 2005
This makes sense because Facebook was one year old in 2005 and OpenAI is 11 years old now. Eleven is just two ones so it’s basically the same thing as one so it is sensible to make that comparison
What is your use case that you see UI lag between vscode and sublime? Honestly, I feel zero difference between sublime/vscode/vi. Vscode arguably takes longer to boot up, but that only happens like once a day so it's not a big deal.
I think this is a lot of "I don't like Typescript/Javascript for serious things" or "Electron sucks" posturing rather than an actual tangible difference.
If you don't feel these differences every keystroke, count yourself lucky to have slower perception or typing, rather than accusing folks of posturing.
Your brain processes (visual) information at a resolution of >= 80ms[1]. The idea that you can tell the difference between 10ms or 50ms latency when typing is simply untrue (both events will appear instantaneous). I say this as someone that has played Counter-Strike professionally and have a sub-200ms reaction time. (Auditory perception is processed at a higher resolution, but the article is decidedly not about that.)
i cannot tell exactly but it kinda bothers me while working/typing
it is not like a huge latency, definitely not like ssh-connection.
to explain better, i usually have pre-defined set of keystrokes i input, so it's not the issue of latency of a single keystroke, rather compounding effect.
another thing is, most of the LSPs, highlighting etc are visibly slower on vscode. I am also having many plugins/extensions so that is partly to blame.
in the recent versions of vscode, they started supporting tree-sitter, which is quite nice in terms of performance.
We do, and the comparison is apt. We are the ones that hydrate the context. If you give an LLM something secure, don't be surprised if something bad happens. If you give an API access to run arbitrary SQL, don't be surprised if something bad happens.
No, that's not what's stopping SQL injection. What stops SQL injection is distinguishing between the parts of the statement that should be evaluated and the parts that should be merely used. There's no such capability with LLMs, therefore we can't stop prompt injections while allowing arbitrary input.
Everything in an LLM is "evaluated," so I'm not sure where the confusion comes from. We need to be careful when we use `eval()` and we need to be careful when we tell LLMs secrets. The Claude issue above is trivially solved by blocking the use of commands like curl or manually specifiying what domains are allowed (if we're okay with curl).
The confusion comes from the fact that you're saying "it's easy to solve this particular case" and I'm saying "it's currently impossible to solve prompt injection for every case".
Since the original point was about solving all prompt injection vulnerabilities, it doesn't matter if we can solve this particular one, the point is wrong.
> Since the original point was about solving all prompt injection vulnerabilities...
All prompt injection vulnerabilities are solved by being careful with what you put in your prompt. You're basically saying "I know `eval` is very powerful, but sometimes people use it maliciously. I want to solve all `eval()` vulnerabilities" -- and to that, I say: be careful what you `eval()`. If you copy & paste random stuff in `eval()`, then you'll probably have a bad time, but I don't really see how that's `eval()`'s problem.
If you read the original post, it's about uploading a malicious file (from what's supposed to be a confidential directory) that has hidden prompt injection. To me, this is comparable to downloading a virus or being phished. (It's also likely illegal.)
The problem here is that the domain was allowed (Anthropic) but Anthropic don't check the API key belongs to the user that started the session.
Essentially, it would be the same if attacker had its AWS API Key and uploaded the file into an S3 bucket they control instead of the S3 bucket that user controls.
SQL injection is possible when input is interpreted as code. The protection - prepared statements - works by making it possible to interpret input as not-code, unconditionally, regardless of content.
Prompt injection is possible when input is interpreted as prompt. The protection would have to work by making it possible to interpret input as not-prompt, unconditionally, regardless of content. Currently LLMs don't have this capability - everything is a prompt to them, absolutely everything.
Yeah but everyone involved in the LLM space is encouraging you to just slurp all your data into these things uncritically. So the comparison to eval would be everyone telling you to just eval everything for 10x productivity gains, and then when you get exploited those same people turn around and say “obviously you shouldn’t be putting everything into eval, skill issue!”
Yes, because the upside is so high. Exploits are uncommon, at this stage, so until we see companies destroyed or many lives ruined, people will accept the risk.
That's not fixing the bug, that's deleting features.
Users want the agent to be able to run curl to an arbitrary domain when they ask it to (directly or indirectly). They don't want the agent to do it when some external input maliciously tries to get the agent to do it.
Implementing an allowlist is pretty common practice for just about anything that accesses external stuff. Heck, Windows Firewall does it on every install. It's a bit of friction for a lot of security.
But it's actually a tremendous amount of friction, because it's the difference between being able to let agents cook for hours at a time or constantly being blocked on human approvals.
And even then, I think it's probably impossible to prevent attacks that combine vectors in clever ways, leading to people incorrectly approving malicious actions.
It's also pretty common for people to want their tools to be able to access a lot of external stuff.
From Anthropic's page about this:
> If you've set up Claude in Chrome, Cowork can use it for browser-based tasks: reading web pages, filling forms, extracting data from sites that don't have APIs, and navigating across tabs.
That's a very casual way of saying, "if you set up this feature, you'll give this tool access to all of your private files and an unlimited ability to exfiltrate the data, so have fun with that."
I do think that local-first will end up being the future long-term though. I built something similar last year (unreleased) also in Rust, but it was also running the model locally (you can see how slow/fast it is here[1], keeping in mind I have a 3080Ti and was running Mistral-Instruct).
I need to re-visit this project and release it, but building in the context of the OS is pretty mindblowing, so kudos to you. I think that the paradigm of how we interact with our devices will fundamentally shift in the next 5-10 years.
[1] https://www.youtube.com/watch?v=tRrKQl0kzvQ
reply