Vault configuration best practice

Questions / Help
, , ,
1

Hi Friends,
I am about using libvault to read DATABASE_URL and other secrets.
I m going to use Vault.Auth.Kubernetes
I am not sure I am following mix release guide and ConfigProvider
Should I define a module VaultConfigProvider with codes like

def init(vault_config) when is_list(vault_config) do
    vault_config
  end

  def load(config, vault_config) do
    # We need to start any app we may depend on.
    {:ok, _} = Application.ensure_all_started(:jason)

    {:ok, vault} =
      Vault.new(
        engine: Vault.Engine.KVV2,
        auth: Vault.Auth.Kubernetes,
        http: Vault.HTTP.Tesla,
        host: host
      )
      |> Vault.auth(%{role: role, jwt: jwt})

{:ok,
 %{
   "SECRET_KEY_BASE" => "..",
   "DATABASE_URL" => "ecto://dbhost:5432/.."
 } = json} = Vault.read(vault, "myapp/env")

Config.Reader.merge(
      config,
      my_app: [
        some_value: json["DATABASE_URL"],
        another_value: json["DATABASE_URL"],
      ]
    )

and then remove all lines that get system env from releases.exs?
Is there the easiest way to get the vault secrets?

Where is the best place to define vault_config_provider.ex?

2

If you want to use Vault for the DB connection, then you should probably use :configure option from DBConnection to fetch secrets for each connection just in cause if one of the connection dies. Otherwise you can end with outdated credentials and cause application to crash in such case.

If you use such approach, then you do not need releases.exs nor config providers, as everything will be handled during runtime. The “disadvantage” (which is no problem there) is that each connection to the DB will use different credentials (unless you do some caching for that).

3

So I decided to use runtime. exs in mix release

config/runtime.exs now looks like:

import Config

if config_env() == :prod do
{:ok, _} = Application.ensure_all_started(:jason)
{:ok, _} = Application.ensure_all_started(:hackney)

jwt =
  System.get_env("JWT_TOKEN_PATH")
  |> Kernel.||("/var/run/secrets/kubernetes.io/serviceaccount/token")
  |> File.read!()

vault_host = System.fetch_env!("VAULT_ADDR")
vault_k8s_role = System.fetch_env!("VAULT_K8S_ROLE")
vault_prefix = System.fetch_env!("VAULT_PREFIX")
vault_env_path = System.get_env("VAULT_ENV_PATH") || "secrets"

{:ok, vault} =
  Vault.new(
    engine: Vault.Engine.KVV2,
    auth: Vault.Auth.Kubernetes,
    http: Vault.HTTP.Tesla,
    host: vault_host
  )
  |> Vault.auth(%{role: vault_k8s_role, jwt: jwt})

{:ok, vault_secrets} = Vault.read(vault, "#{vault_prefix}/#{vault_env_path}")

################################################################################
## Release Config (with Vault secrets)
################################################################################
config :myapp, MS.Repo,
  # ssl: true,
  url: Map.fetch!(vault_secrets, "DATABASE_URL"),
  pool_size: String.to_integer(vault_secrets["POOL_SIZE"]) || 10
...
end

and to include runtime.exs on release start runtime_config_path should be added to mix.exs:

  releases: [
        myapp_web: [
          runtime_config_path: "config/runtime.exs",
          version: "0.0.1",
          applications: [
            myapp_web: :permanent
            ...

So no additional config readers were used

2 Likes
4

Thank you for sharing the code Marina. I think it mostly similar to what I had in mind when I asked the question and likely will fallback on this definitely a good answer.

I have rewritten my solution as a configprovider with the idea of using an agent for vault to do later dynamic secrets and updates of tokens, but I am seeing the currently my configprovider is not starting with a simple iex -S mix phx.server only when doing a full release.