How to access environment variables?

Questions / Help
1

I’m working on making my dotenvy package a bit easier to work with. In particular I’m trying to support using 1Password, SOPS and the like. I have avoiding having dotenvy do any shell parsing of env files – that seemed to potentially be a can of worms with more security considerations than I wanted to deal with in the core library. But sometimes it’s really handy to run bash commands in there, e.g. using the 1Password CLI tool:

export MY_PASSWORD=$(op read op://MyVault/SomeThing/password);

I thought I’d be able to source a .sh from within the runtime.exs config I’d have the necessary environment variables… but it turns out things are more subtle. Although you can do something like

System.shell("source secrets.sh")
# or
System.cmd("zsh", ["secrets.sh"])

it turns out that any environment variables exported in the secrets.sh file are unavailable because they are scoped to that one system thread (? not sure if that is technically correct). Once the command completes, those variables are no longer accessible, so System.get_env("MY_PASSWORD") or the dotenvy convenience function env!("MY_PASSWORD", :string) come up empty. The return value of these bash scripts aren’t super useful either since they may contain lots of different variables and usually you only get the value of the last line returned.

So far, the only simple workaround I’ve found is to do a helper script that exports the necessary variables and then launches the Elixir app, e.g.

#!/bin/bash
export MY_PASSWORD=$(op read op://MyVault/SomeThing/password);
iex -S mix

This works because the variables are exported to the same scope as the system thread running the Elixir app (again, forgive me if my technical explanation here isn’t accurate – but things work because the scope of the variables is the same as the Elixir app).

Is there some more elegant way to do this? I could come up with a convenience “alias” for each mix command that would guarantee that the proper env vars were available, but that would mean having to make a new bash script each time you needed to support a new mix task.

Perhaps a different option would be to stream the output of the command into a file and then parse that? If I end the secrets.sh file with a simple call to env, I will get ALL the environment variables returned as a string, which I can then pass to Dotenvy.Parser.parse/3 and I have everything I need… that might be easiest, but it’s admittedly weird to end your bash file with a call to env…

Any ideas/thoughts/feedback welcome! Thanks in advance!

1 Like
2

How about doing somethings like this:

System.shell(~s'bash -c "source secrets.sh && export"')

and then parsing the output and calling System.put_env on that?

3 Likes
3

Brilliant! Thank you! I had to use && env to get an output format that I could use, however – I couldn’t get System.put_env/1 to work, but I could do this:

{raw, _} = System.shell(~s'bash -c "source envs/secrets.sh && env"')
{:ok, system_env_vars} = Dotenvy.Parser.parse(raw)

source!([
  System.get_env(),
  "#{env_dir_prefix}.env",
  "#{env_dir_prefix}.#{config_env()}.env",
  "#{env_dir_prefix}.#{config_env()}.overrides.env",
  system_env_vars
])

By using the parsed output system_env_vars instead of System.get_env(), all the environment variables have the values that I would expect, so does what I need!

4 Likes