How to avoid duplicated build time vs releases configuration

(I have found some threads in the forum that in one way or another touch this topic, but none of them solves this question in a clear way.)

As you know, build time configuration is located in config/config.exs, Mix uses that file to generate the application resource file, which contains the hard-coded application environment, among other things.

On the other hand, we have now config/releases.exs, which is evaluated at runtime, but only for releases.

Now, in my project everything is set via system environment variables regardless of the type of server. So, for example I have

config :avrora, registry_url: System.fetch_env!("AVRO_REGISTRY_URL")

and a dozen others like that.

Is it correct that despite the intended uniformity, I have to repeat that line in config/config.exs and in config/releases.exs?

Is there a way to avoid that duplication? I was thinking about creating config/common.exs, to be imported in both files, but the docs for mix release say you cannot use import_file (does it mean import_config?).

1 Like

If it suits your use case you can call import_config("releases.exs") from inside config/config.exs, which would allow you to only specify it in one file.

It doesn’t work for me, because there is production stuff that it is not common to the rest of environments.

I have come out with a solution, it is not very elegant, but seems to work correctly in my proof of concept.

First of all, the new Config module is unrelated to Mix, which is the point. Therefore, I argue that it should be possible to invoke import_config/1. Granted, there is a MUST NOT in the docs, but I don’t see it justified looking at the source code. Perhaps it is being too generic assuming a common Mixish invocation pattern?

Secondly, the priv directory is included in releases and available in a robust way via :code.priv_lib/1.

Therefore, we can have a hybrid approach. In config/config.exs you write:

import_config "../priv/common.exs"

and in config/releases.exs you write:

import_config Path.join(:code.priv_dir(:my_app), "common.exs")

Any objection to that approach?

Yeah, that would have been too easy :slight_smile:

Someone else will have a more qualified answer than me, but I know configuration was thought long and hard about for the inclusion of releases in Elixir 1.9, and I think there was a good reason why they did it this way - especially since they call it out so clearly in the docs.

Well, since import_config is not allowed in config/releases.exs currently, I don’t think this is very practical.

In earlier projects (using Distillery 1) I opted for using the Conform approach. Here, you have to maintain the release configuration in an entirely different format, and then define how that is transformed into Elixir terms - which is a lot more overhead than maintaining config/releases.exs, but also gives a cleaner/stricter interface to configuration. It should be similar to using e.g. a YAML configuration provider with 1.9 releases.

I only have a small app that uses the new Elixir 1.9 releases, and in this app, there’s very little overlap between the Mix config and the config/releases.exs. I currently hardcode configuration in the Mix config files, and allow a few variables to be specified as environment variables that are read at runtime in config/releases.exs. So far it has worked fine for me, but as I said this application is quite small.

Lately, I prefer using the application environment as little as possible - only using it for the configuration that I cannot specify/change after my application has booted. This preference comes from excellent blog posts like this one by @sasajuric. I don’t use that approach exactly, but the ideas definitely informed the way I tend to do configuration today. Also, libraries like Specify or Vapor are pretty interesting.

I am looking for validation/refutation in this pull request.

I’m the author of Vapor so obviously I’m biased. But my preference is to completely avoid all of the config/**.exs files for runtime configuration. Its way more trouble then its worth. Vapor is a formalization of the patterns that I’ve been using to manage configuration values in the services I maintain. The library itself has gone through a few different iterations but overall its been useful.

1 Like

Vapor seems interesting would you have a tutorial that shows how to use it in mix context,
when starting the application with

iex mix phx.server

and in release context

app/bin/app start

?

Exactly. There is a gap between config/config.exs and config/releases.exs, because one is compile-time and the other is runtime. That’s why it has been historically awkward to do runtime configuration for Mix. config/releases.exs solves it, but only for releases. We are aware of the issue and we have a proposal in draft, to be included on v1.11, that makes it clear what is compile-time and what is runtime. This should be the last stop in our journey to improve Elixir configs that started back in v1.9.

12 Likes

I’m working on a blog post about how we use vapor which will have more detail. But all we do is specify our configuration as a function in our Application module. The function is typically named config!. The function calls Vapor.load! with the configuration providers. The first function we call in app start is config!. Once we’ve loaded everything, we can pass values down to our children processes.

The main benefit to this approach is that it skips configuration files altogether. Loading configuration is identical whether we’re in dev mode or production mode.

Not sure if useful information, but to deal with wanting runtime configuration outside of a release, for stuff like development shells and running tests, rebar3 simply duplicates the support for sys.config[.src] that running a release has.

So I think the equivalent would be if iex -S mix and mix test were able to load application environment from config/releases.exs.

Use confex and use a config.exs and test.exs - it resolves ENV vars at runtime rather than having all these things resolved at compile time.

1 Like

Is the proposal on the elixir-lang-core list or in this forum somewhere? I’ve searched both places and found very long threads but couldn’t find v1.11 draft plans to improve this. Trying to decide whether to adopt one of the nice community runtime config systems or wait for v1.11.

It is still in draft. It should be published in the upcoming week.

3 Likes