I think it was discussed before not only a single time, however I was unable to find any relevant information using search on elixirforum.
Let’s say we talk about projects containing some kind of secret values, both present for deployed environments and dev. Here are a few ways I’ve usually seen them being set:
Hardcoded for dev in form of a file that is gitignored, for example: dev.secrets.exs, for deployed environments just fetched as a env variable;
Similar to 1st approach in terms of structure, but generated semi-automatically using a mix task or something similar;
Everything set as a environment variable in runtime.exs and using something like direnv to manage them per-project.
I specifically would like to know more about third option, as it seems that overall it makes the configuration more homogeneous in theory. I’ve seen it being implemented in a few recent projects I had to work with and I didn’t like it very much, but once again this might be an issue of implementation.
Any thoughts on this, what is your personal preference?
what I usually do is somewhat the third option but not exactly what you described.
in dev.exs I use the same envvars as runtime.exs but with sensible defaults(like defaulting postgres to localhost, etc). In runtime.exs i check if the apps is compiled for mix env dev or prod, if it’s prod we change to use the envvars but without the defaults, after all we want it to fail if the value is not set.
i’ve never used something like direnv in production settings, just for dev/local. in my last job we used hashicorp’s vault service, it’s quite handy and having versioning in envvars is great.
but in general i’d go with the thing that fits better your deployment system.
While the project is small, env vars and System.get_env! in runtime.exs is quite enough. (And $MIX_ENV.secret.exs as you said, that complements this model nicely.)
The moment you find yourself juggling 15+ env vars and having to parse and validate their values is the moment you should drop all of it and just use Vapor. It’s easy to cringe at its “enterprise”-y offering but I used it in two projects and once you get over the initial learning curve (where you declare the exact location of all your configurable pieces) it just vanishes from view and does its job well. We used it to get data from env vars and YAML files. But I know it can do more than that.
I think the question is more about dev environments, or better said about how to structure the configuration for both dev and prod in a way that they don’t fight each-other, as on prod the environment those variables are set by the deploy tool.
Do you use any additional tooling (for values that cannot have a default) for initially setting the config? I think usually you would do by having a .env.example file but who knows, maybe you have a better way?
I was thinking that this would be an overkill, however I know for a fact that I will need validation part like really bad very soon for a specific set of projects, so I think it might make sense to adopt it.
The big question is: does it support custom validation functions? We have some very adventurous developers, where they like sitting hours and debugging issues, where they set env variables to something like this: option-1:option-2:option-3, so a full string parse and validation is in order for such values.
I use the 3rd option + confispex library. The idea is simple - you describe a schema for the ENV variables and then just use Confispex.get("MY_VAR") instead of System.get_env("MY_VAR") in runtime.exs. I’ve built this library when our configuration was really hard to manage. Currently, our the biggest project has a config schema with ~260 defined ENV variables. Works pretty well and totally manageable.
Vapor was created before runtime.exs was introduced in Elixir, thus it is more complex.
While I haven’t had to use a library like confispex I think this is the way to go. Instead of enforcing how env variables are passed – that might always be subject to change and inconsistency – expect the input to be there in any which way the surrounding context wants to do it.
E.g. with a new machine I’ve for now opted to use direnv + 1password cli. I might eventually switch to a more dedicated secrets store over 1password and it’s unlikely that I’ll ever run direnv in production. But I’m certain that I can somehow provide the necessary data as env vars anywhere.
That doesn’t mean one cannot additionally expect devs in development to use a shared tool to get access to secrets and provide them to the app the same way, but I favor making that an independent decision.
