I am pretty new to Elixir. I have been reading some books and articles and now I want to work on some project to settle the concepts.
I have a project based on microservices, that I would like to replicate in Elixir.
I thought umbrella projects were a good way to go as services could be almost mapped to applications, but there is something I am not getting.
Kafka is the entry point for the application and I thought on using Kaffe as library to handle it.
I need to “consume” multiple topics with different information so different services handle the messages but, as all the apps have to use the same configuration I do not know how to configure Kaffe differently for each application.
It seems it is a common pattern to use apps instead of libraries in Elixir so, Is there any way to define different configurations for Kaffe?
There are multiple things here:
I feel there are things I do not get regarding Umbrella projects … What are the useful for? I have been looking for documentation regarding this type of projects but found nothing interesting.
Application configuration: All the applications inside an umbrella project use the same config.exs file, but each one should define its own dependencies and applications that should be up & running for them to work, but they should share the dependencies … I do not get why I cannot define the dependencies directly on the root project
I like the way umbrella projects allow me to have different applications while having just one repo and be able to deploy everything using releases, but:
I need more information regarding how and when to use umbrella projects
I do not know how to read custom configuration and how to configure an application I depend on differently for each application in an umbrella project.
Sorry for the loooong post.
Thanks in advance your time, any information will be really appreciated.
I would suggest avoiding configuring your app via config.exs - this is a build time configuration. Granted, some libraries make it harder than others, but you should generally favour runtime configuration. Seems like Kaffe supports that with start_link/4 - pull it from ENV, not config.exs.
Umbrella projects are a convenience to help you organize and manage multiple applications. While it provides a degree of separation between applications, those applications are not fully decoupled, as they share the same configuration and the same dependencies.
The pattern of keeping multiple applications in the same repository is known as “mono-repo”. Umbrella projects maximize this pattern by providing conveniences to compile, test and run multiple applications at once.
If you find yourself in a position where you want to use different configurations in each application for the same dependency or use different dependency versions, then it is likely your codebase has grown beyond what umbrellas can provide.
I think you’d be good with just a common lib and multiple projects.
@stefanchrobot Thanks for the hints … I think I will go with the multiple projects aproach. I am now taking a look into distillery to try to figure how to create releases with multiple projects … if that’s even possible.
Regarding configuration, and sorry if the question is too dumb or basic but I am just starting to learn the Elixir environment, by favour runtime configuration you mean to use a config/releases.exs file and read values from environment variables there?
I was wondering if there isn’t any way to load configuration in Elixir similar to configmaps in k8s, that is from a file.
What would be the benefit of creating a release with multiple projects? I’d go with a release per project and would also suggest Elixir’s releases as they are simpler.
config/config.exs and friends (except for config/releases.exs) are build-time configuration. So if you read ENV there, it’s going to be baked into the app with whatever the value was at the compile stage. This is where config/releases.exs comes in. The problem is that it’s used only by an app packaged as a release.
My personal approach and preference is to have the same flow in all cases (prod, dev and test). So my configs are mostly empty, except for things that are required to be configured at build-time. My release config is empty as well. I pushed all config to relevant places much like as suggested by Saša Jurić in his blog post. For dev and test I’m populating ENV using .env files, in prod the ENV is populated by the platform (Gigalixir, Render, etc.).
