I’ve been working with Elixir/Phoenix to replace some Python services and systems. I’m struggling to grasp the best philosophy around structuring applications and deploying. A lot of tutorials and books cover how to write a phoenix all-in-one app, but I haven’t found any good information on structuring multi-app systems.
I have an application that I would like to write in Elixir, it has some business logic, an API, and also a kafka consumer. Typically, in Python, I would deploy the same code (using docker) with different entry-points so that I could scale the api and consumer independently. So the api would sit behind a load balancer, and the consumer would be separate and scaled separately.
I haven’t played too much with umbrella projects, but this seems like maybe the right way to do what I want here. I can’t tell whether I’m working against the recommended Elixir/Erlang ways though.
Should I use umbrella projects and run the specific applications independently, or should I be looking to use an entirely different approach (like run&scale as a single app, or use multi-node clusters, etc)?
If I use umbrella projects, would the idea be to push the ecto models and business logic into a separate app, and then have an app for phoenix, and an app for the consumer?
If I use umbrella projects, would the idea be to push the ecto models and business logic into a separate app
Umbrella apps are for something different, avoid using them if you don’t know what they do yet!
I have an application that I would like to write in Elixir, it has some business logic, an API, and also a kafka consumer. Typically, in Python, I would deploy the same code (using docker) with different entry-points so that I could scale the api and consumer independently. So the api would sit behind a load balancer, and the consumer would be separate and scaled separately.
Elixir and it’s VM loves big applications, that is because the concurrency model scales so well, you can scale them naturally instead of using third party tools like k8 to manage scaling.
There are a lot of approaches and things to take into consideration, however I would recommend you to start with a single application and if that doesn’t work as well as you expected to separate it into different services, because it will not only save you time with infrastructure management at the beggining, but it might prove times more efficient than having services from getgo.
I saw the phoenix supports generate with --umbrella which automatically separates the interface from the business logic, and that suggested that perhaps this was a recommended approach. But I’ve also seen a lot of folks building much more monolithic and deploying as a big distributed cluster (which I think you’re advocating for here).
Umbrella applications are for separating your applications in terms of erlang applications, they will be part of the same runtime, however have their separate runtime possibilities(in simple terms).
I saw the phoenix supports generate with --umbrella
Yes you can do that, however first of all there is no need for that as all processes and functions are isolated in the system by design, the only thing global in an elixir application is the configuration, and you can separate configurations should you need so. The second reason is that umbrella applications introduce a lot of unnecessary complexity, witch you don’t need without a good reason, especially if you are not very familiar with the ecosystem.
