What other device would you suggest as a home server that a non tech person can set up themselves and has enough power to run several Chrome tabs? Access to iMessage is a plus. Small beeline Windows devices could also work but it’s Windows 11, slow as molasses.
This kind of research is underrated. I have a strong feeling that these kinds of harness improvements will lead to solving whole classes of problems reliably, and matter just as much as model training.
Thank you for this. I'm in Europe with an established SaaS that's been running in production for years and I've converged on a similar stack (OVHCloud instead of Hetzner). However, I've realized you can stay sovereign and independent in any jurisdiction (not just Europe) just by simplifying your stack and running a few baremetal servers in-house.
Just buy a few Mac Studios and run them in-house with power supply backup and networking redundancy and you're good to go to serve more than 10k - 100k requests/second which is good enough to serve a million customers. You don't need VMs: a single Mac Studio gets you 2–4x the power of m7i.2xlarge on AWS, and pays for itself within a few months of AWS bills. You can do local AI inference and get Claude Opus-level performance (Kimi K2.5) over a cluster of Mac Studios with Exo.Labs (an unofficial Apple partner). You get free S3-compatible object storage with zero ongoing storage costs with MinIO (yes it's redundant even if you lose a server, and your hosting provider can't hold your data hostage by charging for egress). Postgres runs like a beast and is incredibly easy to setup - you get zero latency DB because it runs on the same machine, has access to lots of RAM and you're not paying per-GB or per-core. Managed databases are a scam. You don't need an Auth provider, just do passkeys yourself. And the great thing about Apple Silicon hardware is that it is amazingly quiet, reliable, and efficient - you can do thing like run headless browsers 3x faster and cheaper than on standard server hardware because of the unified memory and GPU acceleration, so you're not paying for CI/CD compute by-the-minute or headless browsers either.
This entire stack could give you computing power equivalent to a 25k euro/month AWS bill for the cost of electricity (same electricity cost as running a few fridges 24/7) plus about 50k euros one-time to set it up (about 4 Mac Studios). And yes, it's redundant, scalable, and even faster (in terms of per-request latency) than standard AWS/GCP cloud bloat. Not only is it cheaper and you own everything, but your app will work faster because all services are local (DB, Redis cache, SSD, etc.) without any VM overhead, shared cores, or noisy neighbours.
I, too, once believed this. Then I had the displeasure of watching a $10,000 server fail during Christmas travel (about 20 years ago now). A single RAID drive failed. Then, during the rebuild, a second drive failed. Then the RAID controller itself failed catastrophically, losing all the RAID volume metadata. When we restored from backup, we discovered that the sysadmin who had just quit a few weeks before had lied to us about the backup system, and we had no backups.
This is the sort of black swan event that happens every 5-10 years. It's an unusually bad event, even by black swan standards, but stuff like this happens.
The fundamental problem of self-hosted databases is that you test the happy path every day, but you only test true disaster recovery every 5-10 years. And in practice, this means that disaster recovery will usually fail.
With a managed database service, most of what you're paying goes to making sure that disaster recovery works. And in my experience, it does. I've seen RDS database servers fail catastrophically, and completely rebuild in under 15 minutes with virtually no data loss, with almost no human intervention at all.
If you care about your customers' data, I think that a reputable managed database is the right move until roughly the point that you can pay for a full time database administrator. At that point, sure, roll your own. But do regular disaster recovery tests, lest you discover that a recently departed DBA has been lying to you.
Yeah but even with managed database services you don't know if your provider has invested into proper testing of their recovery so you have to test it anyway. Major services like DigitalOcean have been known to shit the bed with your backups. If you don't test your backup recovery, you don't know if you're screwed even if you're paying for "managed" services.
I test my backup recovery several times a month by actually baking into our CI/CD workflow under certain conditions. The entire production database gets restored from backup every week.
You could use a managed db service as a live replica dedicated as a backup only. The queries would go to your local database on beefy hardware, while the replica would just have to be powerful enough to keep up with the WAL stream.
I've designed our app so that there are only two stateful services that matter: Database and Disk. Everything else is cattle, you can shut down or spin up new instances and the load balancer redirects requests with no impact. Making Postgres redundant is a matter of careful configuration with PGBouncer + HAProxy + Patroni. However for a long time we had a much simpler setup: just restore a new database from backup on a new machine if the main one failed (one-time simple script run manually - not automatic, means a little bit of downtime if there's a failure, but it worked). Or you could use CockroachDB. Making disk redundant: just use MinIO for S3-like disk (that's also where DB backups are stored). You can lose up to 2 out 4 of your servers and you lose nothing.
With this setup if 1 or 2 Mac Studios fail (or need to be restarted for updates) everything just keeps running smoothly with no customer impact. It also helps that the app itself is on the Elixir BEAM (Phoenix) so everything "just works" across all machines.
Do note MinIO is deprecated and no longer maintained, discussed here[1]. There are plenty of alternatives though, most mentioned in the referenced submission.
MinIO was a previously open source blob store. It's pretty old, it was basically created right around the time S3 took off.
You should probably reconsider going with it in 2026 unless you're fine with their new (non -opensource) offering. It still has a "free" license, so it might still be an option depending on your priorities.
But there are alternatives around, some being arguably much easier to run/maintain for small deployments like this.
Yes, except for one HAProxy server. The setup I described isn't fully in production yet, but my testing confirms it works. We've been running for years on one single baremetal server on Hetzner/OVH though. And macOS makes sense for one of our main workloads (headless browser agents). Much better than browser-in-linux-docker for many reasons.
I’m not anti American, that’s not the main point of my setup. The main point is I want to own it, not rent it. Apple doesn’t control my production setup after it’s in my hands. Macs from 10 years ago still work.
It has the standard property of ownership: nothing gets turned off without YOUR permission, or at minimum legal proceedings in the area where you are located.
I'm not aware of any standard of property ownership with regard to Mac OS, Windows or any other proprietary software. The end user is granted a license to use the software. That license can be revoked at any time for any reason.
HP makes them, so does Dell. They cost a bit extra, but essentially the whole Federal government runs on nothing else.
The difference between EU and US is that it's possible to make all components in the US, using US equipment, and so some companies do because it commands a pretty decent premium. It's not even that hard since most components (e.g. reference motherboard designs) are still designed and actually built in the US. China still really mostly does what you might politely call "commercializes US tech". And let's not discuss too deeply if they correctly pay licensing for all the components they make, because nobody enjoys that discussion.
And yep, as you might expect, only Intel chips, no Nvidia cards ... and that's not the end of the limitations. The previous version had no USB-C monitor support, never mind one USB-C cable to multiple monitors, but last year intel really pushed a bit harder. But even this year, I'd hope you're not going to be trying to use these machines for gaming.
The EU can't even make a modern motherboard's USB port chip.
Oh and yes, there are cracks in the US version too. The phones used, for example, are iPhones. Radio designed in South Korea ...
I'm rather curious where in the US HP and Dell source, let's say, their displays?
And while many (but certainly not all) of the other components could be made in the US, it's expensive and capacity is limited. So even the likes of HP and Dell have most of it done in Asia. Even Intel chips generally pass through Asia for assembly and testing, and their modern CPU tiles are likely to include TSMC-fabricated components.
All this is to say: the US is not tech independent (unless ancient tech counts). No single country is.
Though if you're just trying to say that the EU is significantly more tech-dependent than the US then I agree of course.
The most technologically critical component of ASML's EUV lithography machines (the EUV light source) is designed, developed, and manufactured in California by Cymer.
> forces the Dutch government to put export controls on some of their machines
That's because the critical EUV light source technology is developed in California by a US-based subsidiary of ASML. The US and EU have mutual interest in protecting the technology and machines. If export control agreements were not in place then ASML would have never been permitted to acquire Cymer. And if they are not enforced then the US would almost certainly require ASML to sell Cymer back to US ownership, TikTok-style.
>However, I've realized you can stay sovereign and independent in any jurisdiction (not just Europe) just by simplifying your stack and running a few baremetal servers in-house.
Only if you have physical offices and staff in every jurisdiction you're serving.
Presumably you have a home where you live? That's your physical office. And no you don't need a presence in every jurisdiction you serve. Visa payment network serves the world from the US.
Yes, but not where my customers live. The whole point of "sovereignty" is to serve customers from a location that is bound by the laws of _their_ jurisdiction, not mine.
There are quite a few factors that matter. The place where data processing and storage takes place is one of them.
It matters who can physically take control of the servers. It matters where the encryption keys are stored. The storage and processing location also matters for compliance with data residency laws.
But it's not the only thing I mentioned. Having physical offices and staff in a jurisdiction usually goes along with setting up some sort of legal and taxable entity that has personally responsible directors.
I have been self hosting since couple of years, yes I got very very interested in self hosting my apps, away from the cloud overlords, but the major issue is the network.
You'll need business internet plans with redundancy and based on locations that might be prohibitively expensive. Some startups might even require their own AS numbers.
Also the connectivity to the data centers or cloud infra like WAF , CDNs etc will be definitely worse compared to cloud instances. Then comes firewalls, their configuration and their redundancy.
These things will matter if you're serious about your SaaS.You could definitely co-locate, but that's another cost, then comes the redundancy of everything, from servers, to disks to network (routers and switches etc).
I personally believe that modern hardware is pretty reliable and doesn't need redundancy in every layer, but most people won't agree with and when startups have enough money, this doesn't matter to them.
I think the only reason the common public is unable to start SaaS is handling and managing these problems. Redundancy costs a lot. And many startups don't want to deal with it even if it'll help them in long run. They just gather enough cash and throw at the overlords.
I do hope that the general infra should improve so that can properly host their own.
Nevertheless I'm still trying to start something in SaaS space and self host from my home...
Got lucky that we have a good personal relationship with our small local ISP and I trust they handle that for us. In the future I want to make it redundant by getting a second gigabit fibre connection.
Ah yes, MinIO, that open source S3 alternative that got archived last week.
To me that's the biggest problem when self-hosting services.
On day to day operations, some times it just breaks and the time to get it back varies from a couple of hours to a couple of days.
And for the longer term you regularly have to upgrade things yourself which takes time and energy and is stressing for stateful deployment.
And then you have it, at some point maintainers are just exhausted and the project is gone.
Are you actually using Exo for local clustered AI inference? I’ve considered it a few times and keep finding horror stories. Never seen someone report it’s actually working well for them.
Great post, and interesting setup - harkens to days of old, when this was simply how things were done in the first place - but one question that I have, apropos:
>.. serve more than 10k - 100k requests/second which is good enough to serve a million customers.
What is your network connectivity like for this setup? Presumably you operate in a building capable of giving you fiber, with a fixed IP, or something like that?
Gigabit fiber with static IP for about 40 EUR per month. I plan to make it redundant with a second gigabit fiber connection from a different provider but haven’t done that yet.
> Presumably you operate in a building capable of giving you fiber, with a fixed IP, or something like that?
That is not really a rarity these days. I have symmetrical gigabit fibre with a fixed IP here in a Spanish farmhouse 45 minutes from the nearest population centre
In some countries and with some ISPs, you cannot get a fixed IP address at all, unless you register a business and prove to the ISP that you are running a business. I am guessing they will bill you accordingly then, and still have the same shoddy connectivity. I have seen shoddy connectivity with Pyür in Germany for a whole office building. Even as a business you are not immune to bad ISPs.
I guess Spain benefits from having a former national telecom. Movistar charges me a (outrageous by local standards) €30/month for a static IP on my residential fibre
No SLA in the world is going to help in a rural area, when a winter storm brings a tree down on the fibre :D
But they offer the exact same specs to business customers in the nearby town. I appreciate Spain is well ahead of most other countries on connectivity, but I can't picture gigabit + static IP being a dealbreaker in most of Western Europe
“Let it crash” doesn’t mean keep bashing your head against the wall. Elixir makes it easy to write state machines which reason about different types of failures, but it’s more declarative (this process requires X and Y preconditions, otherwise do Z) rather than imperative (I have to try/catch failures due to X and Y, now do Z). With Elixir you can actually specify that the process doesn’t start until the DB connection is ready, if that was the cause of the failure, it won’t start again (something else can take care of the DB). When the LLM API returns an error you can put the agent in a paused “errored” state and then you can have a different process decide what to do with the error, and pass it back to the main agent when it’s done. This is all really elegant functional code in Elixir compared to try/catches and if statements.
reply