I’d love for a language / framework that allows for an application to be composed of “modules” that can either be run in a single process, or deployed as multiple independently scalable processes, with a mostly transparent RPC system requiring minimal boilerplate.
My IDE should be able to easily traverse the call graph. My development environment should be simple to setup.
I’ve worked on microservices that required an insane amount of boilerplate to do simple things. Like 7 layers of controllers, clients, services, data services, etc, just to fetch a simple piece of data. And the developer experience of running dozens of services in a Kubernetes cluster running on my dev machine was awful.
Does anything like this exist?
I only dabbled many years ago but Erlang/OTP comes to mind.
And tRPC for TypeScript calls in browser and server.
I made a framework along those lines for work which we used for many years to mediate ~1000 namespaced commands across dozens of repos. Whether it ran distributed or bundled (I called it "mega-service" mode) was completely controlled by config. We mostly used ZMQ for inter-service messaging, but I had implemented http, redis pub/sub, grpc, etc at various times to prove the concept.
Interestingly, we're pursuing a monorepo & multi-monolith setup for the next version of our platform. So lib-courier is no longer necessary to stitch it all together. It was fun while it lasted though. Once you understood the routing algo & code patterns, lots of stuff "just worked".
The industry has been trying to do "transparent remoting" for decades now. DCOM and CORBA were both basically that.
It turns out that "transparent RPC" is basically a contradiction in terms. As soon as you start doing things across process boundaries, and even more so across network boundaries, it requires a very different approach for API design - something that's very cheap locally, like passing objects by reference, becomes expensive and full of footguns.
If you reduce the feature set to the point where it can be transparently mapped to either local or RPC - which is, basically, function calls processing and returning data organized into arrays & trees (but not graphs) - there's still the issue that RPC has so many more failure points that you have to handle that would never light up in local.
This is all still doable; I have my doubts about practical the end result would be, though.
By “mostly transparent” I was mostly thinking type safety, ergonomics, no hand written boilerplate, etc.
Totally fine to support a limited set of primitives and collections passed by value, and require developers consider the failure modes.
In my experience HTTP+JSON or gRPC are usually lacking in at least one of those respects. If you constrain all the services to be the same language it becomes easier.
Hey, we have a system like that at my company. Microservices are self contained and responsible for exposing their interface via SDKs created with a tRPC like library. Jump to definition works across the entire monorepo like a charm. Could you take a look and give me any feedback? https://github.com/rectech-holdings/umbrella-corp-boilerplat...
My IDE should be able to easily traverse the call graph. My development environment should be simple to setup.
I’ve worked on microservices that required an insane amount of boilerplate to do simple things. Like 7 layers of controllers, clients, services, data services, etc, just to fetch a simple piece of data. And the developer experience of running dozens of services in a Kubernetes cluster running on my dev machine was awful.
Does anything like this exist?
I only dabbled many years ago but Erlang/OTP comes to mind.
And tRPC for TypeScript calls in browser and server.