Hope you are well! Have been playing around with Elixir for a few months, loving the language I’m looking for advice on how to manage dependencies between multiple Elixir applications.
My intention is to build a backend system for a bank made up of many (100s, maybe 1000’s) separate independent applications that call into each other - I like the Elixir approach of building several smaller applications, but I am having some trouble setting up config & dependencies.
I don’t want to go the umbrella application route, since it sounds to me like all config would live in one place - I anticipate each application having its own configuration, and I’d like to be able to release individual applications without releasing a new version of the entire system (open to suggestions on this though?).
Pursuing this so far, I have a single Github repo holding multiple Elixir applications, each generated separately. My intention is that application A can call functions in application B - initially, to make that happen, I added application B as a local mix dependency in A.deps().
What I’ve found is that I have to add calls to import all config needed by application B inside application A, which feels wrong? (ex: A is a CLI tool which calls into B, which has some Ecto configuration and queries to return values; if I launch A, in order for A to call B’s public API functions, I have to import B’s config in A’s config files)
What am I missing here? The independent application approach makes a lot of sense to me and feels intuitive, but I’m not sure how to set it up so that these applications can fluidly call into each other while each application contains its own concerns like config, etc. I’m looking to set up something like a microservices design, where “services” are independent applications all running inside the BEAM.
Umbrella applications have the following characteristics, with the goal of being able to call code from each other, and being deployed either together, separately, or not at all (if they are pure library dependencies).
They share compile-time config.
This is required because compile-time config could allow functions in B to compile differently. You could then not reason about what code you are calling functions in B from A without knowing how B was compiled.
They share the same versions of their deps (though only compile the subset they personally need).
For similar and more obvious reasons. This ensures no application A compiles with an older version of Ecto, that calls B, where B makes calls to an Ecto function that does not exist in the older version.
They can be deployed individually.
Because you can now reason about the compile-time of these interlinked apps, you can deploy all of their code together as separate runtimes, and invoke each other’s functions.
I’d argue that umbrella applications are the right abstraction for what you are trying to accomplish. Poncho projects are an approach to code sharing, but it sounds like you need to reason about configuration and dependency sharing as well. The downside is that compile-time config and deps changes in an umbrella “dependency” requires re-releasing the umbrella applications that depend on it, but that’s the only sane way to reason about such things IMO.
You can still build these umbrella units of code as individual releases per application, configured differently.
One way to do this might be to use mix targets to represent which microservice you are building a release for. If apps Bar and Baz both depend on Foo, you could build their individual releases with MIX_TARGET=bar, for example. Mix deps is aware of targets via Mix.target() and config is aware of them via Config.config_target(), so you could conditionally configure Foo differently by branching on the target.
Another path is to move more individual application configuration out of the compile-time oriented config.exs files and into runtime configuration off of environment variables; then re-configuring your microservices is a matter of changing the value on the host and restarting the application. Vapor is a good tool for this.
To not pile up on the rest of the replies I’ll simply offer you the possibility to make your own internal library projects and just use that in each executable project. So if project Z is the library and projects A, B and C need it then it feels quite logical for each of them to have separate configuration.
That being said, if such a setup is not appealing to you – because indeed copy-paste programming is error-prone – you either should create helpers to inject common configuration or just use an umbrella application.
If you are to be working on the big scale that you mentioned then it’s hard to give you a bulletproof recommendation. Umbrella apps have their own problems however.
I’m looking for advice on how to manage dependencies between multiple Elixir applications.
