Recently, at the Code BEAM Lite Amsterdam, I had very interesting conversations with @sasajuric, @michalmuskala, @voltone, @Crowdhailer and a couple of other people. Amongst the subjects was the concept of configuration, and choices to be made here.

Most importantly, @sasajuric’s talk had some interesting points:

• When you decide to do something yourself (in his talk: building a Continuous Integration system), you can have a very focused implementation that does not need a ‘general purpose’ configuration layer with all bells and whistles.
• Configuration just as passing arguments to functions is infinitely more flexible than config files.
• In a system where everything is done in the same language, there is no difference between DevOps and Programmer.

And on the other hand, the talk by @voltone:

• explained the bad experience of configuring Erlang’s built-in :ssl-library properly. (it’s default configuration is very unsafe).
• mentioned that many of the HTTP(S) client libraries out there either require the programmer using them to pass on the proper :ssl-configuration, or the few that do use a secure configuration by default will completely replace it if the programmer passes in their own.

So, while about a year ago we had a topic talking about how to configure your application and libraries you are building, I think it is time to look at this subject anew.

There are roughly three ways to configure a bit of functionality (whether a library, an OTP-application or something else) in Elixir land:

1. Pass options as argument(s) to a function. This is the most flexible, because it allows calling the function from many places in your code with different options. Elixir makes this approach easier by having keyword-lists as first-class type.
2. Add configuration settings in the config/config.exs, config/some_mix_environment.exs and similar files. This makes it possible to have different configuration on different environments (for instance, on your local computer the database has a different password than on the production machine). However, it is not possible to use functionality that only can be configured like this in multiple different ways within the same environment.
3. Configure your application using environment variables, and use some way to read these in your application’s code. The nice part of this approach is that you can have different settings on different machines (even if they are both ‘production’). The bad part is that you currently need a special way to make sure these are inserted in your application’s configuration.

Problems with (2) config.exs-style configuration and (3) environment variables.

Configuration happens at build time

(2) and (3) have the extra catch that when you build your application, for instance when building a release using a tool like Distillery, the configuration is read during building rather than when ran. This means that you either need to build on the machine that will run the release, or your need to take extreme care to make sure the configuration values are still relevant.

Ad-hoc, turing complete configuration.

Furthermore, values filled in in (2) and (3) very much affect the functionality you use in an implicit and ad-hoc way: From looking at the code itself it is impossible to see how it is configured (or that it even has special configuration). Also, because config.exs-style files are essentially just normal Elixir files, they might contain a spaghetti of arbitrary Turing-complete nonsense; most commonly some files override some settings of some other files, making it more difficult to see which configuration will actually take place in the final application (without running Application.get_env manually from within the running application.)

Problems with (1) functions that have option-arguments

Now, (1) is definitely the most flexible. Most importantly, we can configure it differently in each and every one of our tests.

No standardized way of checking the current application environment

However, something that does stick out like a sore thumb is that there does not seem to be a standardized way to check inside your code in which (Mix) environment you currently are: Mix.env will not work inside a built application, because the :mix OTP application will not be included.
Instead, it’s documentation actually encourages you to use (2) to configure environment-specific configuration.

Maybe this is something that we could improve?

Options-handling via keyword-args is somewhat clunky, and could be improved

• We cannot match on these in the function header, because keyword args are ‘just lists’ and therefore position-dependent: def foo(a, b, c: d, e: f) will not be triggered when someone calls foo(1,2,d: 3, c: 4).
• In almost all cases, duplicate option entries are irrelevant. However, there is no way to extract a map directly from the options at the end.

So almost all option-handling code goes a bit like this:

• options are passed in as keyword list
• we either combine (in some sensible way) or warn/crash on duplicate entries (this is very frequently ommitted by libraries in the wild!)
• these are then transformed into a map (or sometimes a special options-struct)
• we fill in sensible defaults for the options that were not provided. (which sometimes depend on yet other options that were provided).

I think a library (or potentially, since it is such a common thing to do, either a ‘blessed’ library or even inclusion into Elixir’s standard lib!) that makes it easier to do properly; essentially I a thinking about something similar to Python’s argparse module (for ARGV parsing), where a specification for the options (including field names, descriptions, allowed types and defaults) can be used for parsing, warning/crashing on error as well as generating documentation of the allowed options!

Problems in general

Libraries restrict their user

Libraries currently commonly pick either (1) or (2) (sometimes with the possibility for (3)).

I think it would be a lot better if libraries were to use:

• default values
• overriding these per key with what you configured using (2)/(3)
• overriding these per key with what you configured in (1).

Alas, possibly in part because there is not a standardized way to do this currently, many libraries either restrict you to one of these approaches, or completely their default configuration out the window once you start customizing something.
In certain cases, such as the SSL-client libraries, this can have disastrous consequences unless the user of the library is aware (the configuration behaviour of a library is almost always ‘implicit’ and not mentioned anywhere).

And related: libraries could warn if they encounter options that they do not recognize, but currently usually do not AFAIK.

So, those are my current thoughts on the matter. I think we can improve this current situation as a community. In this post, I gave some suggestions that I currently think might be worth exploring, but I am also very interested in your suggestions and opinions about this matter.