kreiling.io

Manually Load an Elixir Config File at Runtime

I’d like to discuss how to manually load Elixir configuration files at runtime.

Macro Intent

I want to load an elixir config file (something that looks like, for example, config/config.exs) at runtime, and load that into the application configuration.

Micro Intent

Ultimately, I want to write a Config.Provider implementation which fetches the contents of a secret stored in AWS secrets manager. I’d like for those contents to be an Elixir script with import Config at the top, followed by configuration defined the same way you would in a project’s config/config.exs file. I then want to load that configuration into the current (running!) application’s configuration.

Questions & Discussion

Let’s assume that I can load the secret configuration file from anywhere into a binary with an arbitrary Config.Provider implementation (ignore SecretsManager for now). How would I go about compiling it and loading it into the application configuration?
What do you think of this approach to secret management, at least in theory?
What (at least roughly) do you do for secret management in your Elixir and/or Phoenix applications?

My Take

Let’s consider an alternative Config.Provider implementation: instead load a YAML file, parse it, and translate it into a keyword list to merge into the current config. In my eyes that has a few downsides that I dislike:

Added dependency on a YAML parser
Designing how that YAML is structured is not a trivial task
It’s a roundabout way to get to what I ultimately want, which is just to load configuration from a remote location into the current node’s application configuration.

These are my thoughts. Let me know yours!

12 comments

#deployment #configuration #secrets

1 4634 12

2021-10-26 13:26:42 UTC

Most Liked

factoryd

We run in ECS and use param store for our secrets. You can simply set environment variables in your task definition under secrets.

"containerDefinitions": [
      "secrets": [
        {
          "valueFrom": "/param/path",
          "name": "VARIABLE_NAME"
        }
]

from there I put them in prod.exs as configuration.

I know valueFrom supports secret manager too.

EDIT: Pass sensitive data to an Amazon ECS container - Amazon Elastic Container Service

Post #1

factoryd

Oh I’m willing to brainstorm. I’m completely isolated and bored just like you!

I do run as a release.

I actually just created a toy app to investigate possibilities right now.

Post #5

kreiling.io

So I’ve thought about it some more and looked at some docs in the Code module and there’s a warning about evaluating code coming from the network. Dang, seems like loading data from SecretsManager would be a bad idea from a security perspective…

Still - could there be another way??

I want to see if it’s possible. My current solution would be to have the config provider do:

defmodule MyConfigProvider do
  alias ExAws.SecretsManager, as: Secrets

  def init(secret_id) when is_binary(secret_id), do: secret_id

  def load(config, secret_id) do
    %{"SecretValue" => raw_elixir_code} = Secrets.get_secret_value(secret_id) |> ExAws.request!()
    {config_to_merge, _} = Code.eval_string(raw_elixir_code)
    Config.Reader.merge(config, config_to_merge)
  end
end

Post #6

Last Post!

doma_dev

If you check that the code you get from your secret manager is signed with a non-revoked key and you have reasonable degree of assurance in your intrusion detection, evaluating signed code is perfectly fine. World runs on evaluating signed code.

It’s all about threat model, right? If your threat model is “my secret storage got compromised”, odds are, arbitrary code execution on your production servers has already happened by then.

But yeah, awesome thread, we were getting secrets one by one on bootstrap, and are currently moving to evaluating one signed secret instead.

Post #12