Honestly; make some blessed standards easier to use and maintain.
Right now running K8S on anything other than cloud providers and toys (k3s/minikube) is disaster waiting to happen unless you're a really seasoned infrastructure engineer.
Storage/state is decidedly not a solved problem, debugging performance issues in your longhorn/ceph deployment is just pain.
Also, I don't think we should be removing YAML, we should instead get better at using it as an ILR (intermediate language representation) and generating the YAML that we want instead of trying to do some weird in-place generation (Argo/Helm templating) - Kubernetes sacrificed a lot of simplicity to be eventually consistent with manifests, and our response was to ensure we use manifests as little as possible, which feels incredibly bizzare.
Also, the design of k8s networking feels like it fits ipv6 really well, but it seems like nobody has noticed somehow.
I like YAML since anything can be used to read/write it. Using Python / JS / yq to generate and patch YAML on-the-fly is quite nifty as part of a pipeline.
My main pain point is, and always has been, helm templating. It's not aware of YAML or k8s schemas and puts the onus of managing whitespace and syntax onto the chart developer. It's pure insanity.
At one point I used a local Ansible playbook for some templating. It was great: it could load resource template YAMLs into a dict, read separately defined resource configs, and then set deeply nested keys in said templates and spit them out as valid YAML. No helm `indent` required.
It's the "make break glass situation little easier" option, not the main mechanism.
Use a programming language, a dedicated DSL, hell a custom process involving CI/CD that generates json manifests. A bit of time with jsonnet gets you a template that people who never wrote jsonnet (and barely ever touched k8s manifests) can use to setup new applications or modify existing ones in moments.
Nothing wrong with toys, they’re meant to be played with.
If you deployed this to production though, the problem
is not that it’s a toy: its you not understanding what technical trade offs they are making to give you an easy environment.
I can tell you’re defensive though, might be good to learn what I mean instead of trying to ram a round peg into a square hole.
k3s is definitely fine to run in production, hence it's not a toy. You just have to understand the tradeoffs they've made, when you should change the defaults, and where it's a reasonable choice compared to other alternatives.
A toy implementation of Kubernetes would be something like a personal hobby project made for fun or to learn, that comes with critical bugs and drawbacks. That's not what k3s is.
Right now running K8S on anything other than cloud providers and toys (k3s/minikube) is disaster waiting to happen unless you're a really seasoned infrastructure engineer.
Storage/state is decidedly not a solved problem, debugging performance issues in your longhorn/ceph deployment is just pain.
Also, I don't think we should be removing YAML, we should instead get better at using it as an ILR (intermediate language representation) and generating the YAML that we want instead of trying to do some weird in-place generation (Argo/Helm templating) - Kubernetes sacrificed a lot of simplicity to be eventually consistent with manifests, and our response was to ensure we use manifests as little as possible, which feels incredibly bizzare.
Also, the design of k8s networking feels like it fits ipv6 really well, but it seems like nobody has noticed somehow.