Configuration Providers vs. a Single Source of Truth: the battle to ensure valid app config

I wanted to solicit the community’s knowledge/experience/recommendations on this one…

Let’s say you’re working with an application that gets deployed to production and you rely on a configuration provider (e.g. secrets_manager_provider | Hex but it doesn’t particularly matter which one).

The problem this solves is that now you can inject sensitive values directly into your Application’s process dictionary. Yay!

However, this creates a new problem: the Application process dictionary no longer has a single source of truth. We don’t know its state! With the introduction of a custom configuration provider, we no longer have a nice mapping from simple config files to Application values.

If we were relying on just runtime.exs, we could easily enforce that certain values be supplied by the environment, e.g.

import Config

config :my_app, :something, System.fetch_env!("SOMETHING")

And that creates a strong contract with the environment: the app will not start unless it is properly configured. This is the approach I leaned into with the dotenvy and it has worked well.

But with a configuration provider, we can’t take the same approach… we might not even use runtime.exs. So we have to move our “defenses” out of the configuration files and retreat into the application itself to verify that our app is configured properly.

Even if our app favors the use of Application.fetch_env!/2 over get_env/3 that’s no guarantee that it has the values it needs when it starts. With the use of the custom configuration provider, we can no longer guarantee that our Application’s process dictionary is in a good state.

How do we defend against this? We could put some code into our app’s start/2 function, e.g. something like:

  @impl true
  def start(_type, _args) do

    Application.fetch_env!(:my_app, :something)  # <-- blow up before start

    children =
      [...]
   # ...

But that feels redundant.

What are other ways to guard against this? Am I missing a trick here? Thanks in advance!

1 Like

For me, a non-issue, just be careful

I’d choose whatever that is clear and rely on the infrastructure & telemetry to roll back bad deploys — for example: if the app does not start and pass health checks within X seconds in Y tries then fail the container.

Unlikely you’d churn configuration 10 times a day or even 10 times a week/month

2 Likes

That sounds like an issue with the config provider. If it cannot enforce failing if expected values are available I‘d look into adding that functionality or look at alternatives.

Iirc ˋruntime.exsˋ is also consumed by a ConfigProvider, just one included with elixir instead of third party. All the constraints you can put on your system with ˋruntime.exsˋ should be replicatable with a third party config ConfigProvider as well.

How though? With runtime.exs , you have that explicit call to something like

config :my_app, :something, System.fetch_env!("SOMETHING")

It would be one thing if an alternate config provider were taking the place of the config files entirely, but in my experience, the alternate config providers get used in addition to the regular config files. Best case scenario (as far as I understand this) would be to devote similar (i.e. non-DRY) code that would evaluate the output from that other config provider. You end up duplicating your “configuration contract” to vet the values provided by the alternate configuration provider.

Can you say more about that? The problem I’m seeing is that the code that would prevent a bad deploy is in the config files. But when a value enters via a configuration provider, there’s no visibility and no visible code that vets those values.