I think their usecase makes a lot of sense as their workloads consume a predefined amount of ram. As a customer you rent a VM with a specified amount of memory so fly.io does not care about reclaiming it from a running VM.
Depends on if they're using smart memory allocation to keep costs lower, IE, if they can pattern that certain workloads only need N amount of memory at Y time, they can effectively borrow memory from one VM for usage in another that has an opposite statistical likelihood of needing that memory.
This is why paying for dedicated memory is often more expensive than its counter part, because that dedicated memory is not considered as part of pooling.
Generalized oversubscription like that is very challenging if not impossible to do securely, since you want to keep workloads isolated to single tenant numa nodes.
I think it would be easier to just enable KSM (KVM can use it to deduplicate memory and save some RAM on duplicate blocks like people loading same libraries into memory)
It’s a common technique though. I believe it’s called oversubscription, where you rent the same hardware to more tenants hoping they won’t use it all at once.
Fly.io themselves admitted they’re oversubscribed and AWS is doing the same for years now
I don't see the problem. Is this not the value proposition of the cloud? At scale things like block level de-duplication and over-provisioning of compute are more efficient to say nothing of power and cooling. This efficiency drives down cost. As long as your data isn't leaking and you get the hardware you pay for when you need it what's the problem?
And then they learn all hotels are doing exactly same thing. One hotel doing is risk, all hotels doing is industry standard.
Airlines, hotels, restaurants, doctors and so on oversubscribe all the time. Whoever complains are free to move and add to their further disappointments.
Memory (DDR4) is like 1.50€ per GB nowadays. There is no need to skimp on it. The most expensive component is still the CPU if you actually want to do work and not just idle your webserver 99% of the time.
this ignores just how competitive these services have become and how much competition there is in this marketplace. Anything that allows a company to balance user experience with cost of hardware will be evaluated. The fact is this is super common because even though RAM is super cheap, at scale, these things add up. If you as a provider don't have to spend 120K on RAM and can use that on more CPUs or something else, it can mean having a competitive edge over the provider that does have to spend that 120K.
If they were competitive, they would be within an order of magnitude of on-prem, but they are not, they are 5x or more. "Super Common" is a weasel word.
There's the 't' series of instances that offer burstable CPU. AFAIK still 1:1 on memory though, and there's models that allow you to pay to stay un-throttled when using t family instances vs. getting throttled when out of burst credits.
