What is the purpose of MinIO, Seaweedfs and similar object storage systems? They lack durability guarantees provided by S3 and GCS. They lack "infinite" storage promise contrary to S3 and GCS. They lack "infinite" bandwidth unlike S3 and GCS. They are more expensive than other storage options, unlike S3 and GCS.
We use it because we are already running our own k8s clusters in our datacenters, and we have large storage requirements for tools that have native S3 integration, and running our own minio clusters in the same datacenter as the tools that generate and consume that data is a lot faster and cheaper than using S3.
For example, we were running a 20 node k8s cluster for our Cortex (distributed Prometheus) install, monitoring about 30k servers around the world, and it was generating a bit over a TB of data a day. It was a lot more cost effective and performant to create a minio cluster for that data than to use S3.
Also, you can get durability with minio with multi cluster replication.
Consider migrating to VictoriaMetrics and saving on storage costs and operations costs. You also won't need MinIO, since it stores data to local filesystem (aka to regular persistent volumes). See real-world reports from happy users who saved costs on a large-scale Prometheus-compatible monitoring - https://docs.victoriametrics.com/victoriametrics/casestudies...
I can't imagine switching at this point. We spent quite a while building up our Cortex and Minio infrastructure management, as well as our alerting and inventory management systems, and it is all very stable right now. We don't really touch it anymore, it just hums along.
We have already worked through all the pain points and it all works smoothly. No reason to change something that isn't a problem.
But what's the point to use these DIY object storage systems, when they do not provide durability and other important guarantees provided by S3 and GCS?
When you want just the API for compatibility, I guess?
Self-hosted S3 clones with actual durability guarantees exist, but the only properly engineered open source choices are Ceph + radosgw (single-region, though) or Garage (global replication based on last-writer-wins CRDS conflict resolution).
It's great for a prototype which doesn't need to store a huge amount of data, you can run it on the same VM as a node server behind Cloudflare and get a fairly reliable setup going
