I’m building an app which I try to architecure as modular, inspired by microservices. The idea is to have one elixir appliciation for each bounded context. The question then is how to implement communication between application, like API calls.
I can implement a HTTP/JSON API facade for each service or I was thinking using clustering and call through application nodes. I guess this would require having GenServer for handling API calls.
Do you have any experience in such architectures and/or advices ?
If you want to use OTP inter-node communication then this is as simple as using libcluster and erpc. There are few gotchas here and there so I would recommend to read the documentation very well if you plan on using it in production.
My only recommendation in this regard is to scale the application vertically as much as you can, since that will save you a great amount of trouble and OTP scales much better vertically than most of runtimes out there.
You can adopt a modular desing inside a single OTP application. What problem are you trying to solve with microservices?
I guess this would require having GenServer for handling API calls.
If each app has a single GenServer process that does the calls to the other apps you are introducing a nasty bottleneck from the get go. I would not do that uless I have a specific reason to.
I’m specifically tied to microservice, but interested in how to make modular application in OTP. Modular for me, means at deployment time: being able to enable/disable components (Applications, supervisor, … ?). Then several nodes could form a whole cluster allowing application components to be distributed.
For example: one node running business bounded context (repo, Ash, …) . On another node a phoenix web app querying de businees app through cluster.
Does it makes sense ?
Architectures like that can be done indeed, but I’d argue that one shouldn’t go there unless there’s a good operational reason for it. E.g. with recent interest in ML/AI in the community there’s a good reason to move that kind of work to a (small) set fo beefy machines with GPU, because having all webservers be such is expensive. This however is not a tool for domain or code organization.
If you really decide to go forward with this: in terms of network, go for HTTP/2 + gRPC if possible (haven’t checked Elixir support but should be fine?). Less overhead in general.
Outside of that, go for simple tokens that expire in 15 - 60 minutes and have a background worker refresh them some 10-30 seconds before they expire. That works quite fine, I’ve done it.
…but in general I’ll echo other posters: don’t splatter your Elixir code in multiple apps if you can help it. One mega server will work much better.
That’s easily doable via env vars or whatever other configuration mechanism you prefer. Your application can just optionally start / not start certain parts of itself depending on them. In fact many projects already do this with dev / test environments: test env does not start most of its runtime dependencies.
I think in order to advise on this kind of thing, I’d need to know why you want one elixir application per bounded context. And then I’d suggest inverting the question, and asking yourself what parts of your application needs the benefits that you’re looking for, and then apply the solution just to those parts.
For example:
I have an image processing pipeline that needs to be deployable independently so that it can be horizontally scaled independently. Therefore, I will create an application to manage the processes and state that are used to accomplish this.
I would not suggest choosing a design pattern for “all bounded contexts”. It may sound simpler, because you now have a one-to-one mapping between concepts (i.e bounded context == application), but it will also push you towards a place where you have a bunch of extra boundaries/state that serve no purpose.
So using the image processing example (and using Ash as frame of reference), I’d have all of my application logic in a single application(i.e shipped to all nodes of a cluster), but would have the resource(s) for processing an image call into some image processing service code, that perhaps runs only on certain nodes of the cluster, or on some other heterogenous nodes. That code would be just “regular elixir code”.
