I’m thinking about implementing micro-service architecture in our application. I’ve studied a lot of resources.
At the beginning I thought I’d implement it on the basis of processes and nodes, it would be more accurate to write the code of each micro service in a separate file and implement all their communications through messaging. To tell the truth, I had problems and I came across Umbrella Project, but this way solves our problems, but not as I expected.
I think about git sub-modules, but yours suggestion’s is very important.
Umbrella projects are a folder structure to develop related otp applications in a single fashion, mostly to resolve the issues of independent projects like version mismatches. If you’re after source code independence why not simply have multiple mix projects?
Sure, but that should be rather a matter of creating the correct kind of communication interface and deploying stuff onto your infrastructure in a way to let them talk to each other. Umbrella on the other hand is just a fancy word for “one folder for multiple apps”. At runtime there’s no difference between apps created as umbrella and apps created independently.
Also keep in mind, that by default erlang nodes connect in clusters automatically, where each node is connected to each other node in the cluster. Depending on the number of physical nodes you expect to be running on this might not be the best architecture.
Because you are using them as a libraries - not as services. This means that you need to have them all on each node you deploy to.
I recommend to use WebSocket for fast communication between micro-services. You have one always active channel and two-way communication. In this case you could deploy each service to another node and/or mix part of them like:
first node: service A second node: service B and C third node: service D
So you are not limited - you can deploy each service to any node you want and you only need to simply configure each service, so it will know on which node it needs to call another service api.
Why do you prefer source code independence? There’s many benefits of keeping the source code to multiple parts of the entire application/company in the same code-base. Such as the ability to make updates to multiple parts of the system “atomically” (i.e. in one git commit). IIRC Google and Facebook actually just use one huge repository that has the code for all of the projects in the entire company.
I feel like there is a fundamental misunderstanding about how to think about systems within the Erlang/Elixir paradigm at play here, and it’s affecting the core design and thinking about how to structure your application(s) to resolve your problem.
A microservice is a program that is designed to have a small area of responsibility, and thus a small surface area when it comes to its API. It can be launched on the same or different hardware stacks as other microservices, but that is a deployment detail that can mostly be ignored.
A process, be it a GenServer or whatever, is a unit of concurrency within the Erlang/Elixir ecosystem. Schedulers allow processes to also be run in parallel, but even that is a result of the concurrent aspect of processes.
Trying to map these two concepts together, with a process equaling a microservice, is going to lead to problems as they are not the same. A microservice could easily be made up of thousands of running processes, or just one…well two if you count the application supervisor.
Similarly, attempting to do clever tricks with blocking rpc calls, or manually controlling the launch of Elixir applications on different connected nodes–while possible–is just increasing the possible configuration and debugging pain points of a running production system. I also just see it as a lot of added complexity that isn’t needed if you just design your solution differently.
If you aren’t going to make a true stand alone microservice (that communicates in a language agnostic protocol) and want to use the slick Elixir communications tools and interconnected nodes…why hang onto that concept at all and not just dive straight into solving the problem the Elixir/Erlang distributed way from the beginning?