How to reload env vars when restarting a mix release app?

Questions / Help
, ,
1

I have encountered something surprising when restarting a mix release app - it doesn’t pick up new env vars.

$ mix new release_test
$ cd release_test
$ mix release
$ FOO=foo _build/dev/rel/release_test/bin/release_test daemon
$ _build/dev/rel/release_test/bin/release_test rpc 'System.fetch_env("FOO") |> IO.inspect()'

This prints {:ok, "foo"} as expected.

Now I want to change the FOO environment variable and restart:

$ FOO=bar _build/dev/rel/release_test/bin/release_test restart
$ _build/dev/rel/release_test/bin/release_test rpc 'System.fetch_env("FOO") |> IO.inspect()'

But it still prints {:ok, "foo"} - it didn’t pick up the updated env var.

When I look at the OS processes, I see a run_erl and a beam.smp process that have the same PIDs as before the restart. So it seems like it may be more of a reload than actually stopping and starting the process, and maybe those long-running processes only read the env var when they first start?

Is there a way, using the mix release scripts, to restart the app and read new env vars as part of the restart?

1 Like
2

The restart is a restart in the beam understanding of things, not by the way how your operating system considers it a restart.

3

you can’t do that. You need to restart the parent shell that started Erlang process to get the new environment variables in.

4

You may want to load vars directly from a .env file and for that several packages exist:

Vapor, by @keathley, offers you several different ways of using it, not limited to .env files.

Dotenvy also as a blog post about it:

https://fireproofsocks.medium.com/configuration-in-elixir-with-dotenvy-8b20f227fc0e

5

Okay. Do you have any suggestions on how to do a “real” restart, so I can reload the env vars?

I can run _build/dev/rel/release_test/bin/release_test stop but that can take a minute or two, and I’m not sure how to tell when it’s done in an automated way.

Typically daemons will write a pid file somewhere that you can use to reliably check the status. I don’t see anything like that.

I may be misunderstanding here - how do you do that? I can open up a new terminal tab, run the same second command, and get the same result. So, that’s not it.

Interesting… my takeaway from reading all this is that you should not configure Elixir apps using env vars, because once the erlang process is running, that’s it - there’s no way to get new env vars into it.

Instead, you use env vars to configure the erlang process. Then your application reads its own config from a file. Some of those tools you linked look like they’re intended to simulate env vars by reading the config from env-var-like files - but they’re not actually env vars.

I think for production releases, it’s probably important to cleanly separate these, and not load any app config from env vars since that only applies to app start. Vapor looks like it should do the trick there, but reading a config file should be simple enough. Thank you for the pointers!

1 Like
6

You shouldn’t run the release manually anyway but through some init system. So to update the environment of the release, you had to change the environment of the service, and then restart it by the means of the init system.

“Init system” here means not only classical init systems like Init, SysVinit or systemd, but also docker and friends.

YES! Env-vars is in my opinion the worst you can do. Use files whenever possible. Easier to change and introspect.

In my opinion this is true for anything, not only erlang/elixir.

There are so many nice structured configfile formats, but everyone wants to use stringlytyped flat envvars?

7

I do, using FreeBSD services. But it just delegates to the app script and calls restart. So basically the service sets up the env vars and calls restart, not much differently from what I’m showing in my example.

I will look to see how other init systems treat it… I would assume they need to know about the OS pid though.

I would at some point like to put it in a jail and then I can just restart the whole jail. I think that’s kind of like pulling the plug on the machine…

Anyway I will look for examples of mix releases with other init systems. I guess I’d be surprised if they didn’t call the same bin script with start / stop / restart etc but we’ll see.

8

I still suspect that a config file is preferable, and just to use beam reloading. But for anyone interested, here’s a script that successfully reloads the env var:

#!/bin/sh
pid_cmd="_build/dev/rel/release_test/bin/release_test pid"

FOO=foo _build/dev/rel/release_test/bin/release_test daemon

until $pid_cmd > /dev/null 2>&1
do
  echo "Starting..."
  sleep 1
done
echo "Started"

_build/dev/rel/release_test/bin/release_test rpc 'System.fetch_env("FOO") |> IO.inspect()'

_build/dev/rel/release_test/bin/release_test stop

while $pid_cmd > /dev/null 2>&1
do
  echo "Stopping..."
  sleep 1
done
echo "Stopped"

FOO=bar _build/dev/rel/release_test/bin/release_test daemon

until $pid_cmd > /dev/null 2>&1
do
  echo "Starting..."
  sleep 1
done
echo "Started"

_build/dev/rel/release_test/bin/release_test rpc 'System.fetch_env("FOO") |> IO.inspect()'
9

Just to follow up: I did figure out how to lean heavily on FreeBSD’s rc system. The main thing is running the app under daemon(8) rather than using the elixir daemon. This produces a pid file, and so the service can use the built-in stop / restart / status behavior.

The following RC script is auto-generated by ExFreeBSD:

#!/bin/sh
#
# PROVIDE: release_test
# REQUIRE: DAEMON

. /etc/rc.subr

name=release_test
rcvar=${name}_enable
pidfile="/var/run/${name}.pid"
procname="/usr/local/libexec/release_test/erts-13.0.2/bin/beam.smp"

: ${release_test_env_file:=/usr/local/etc/release_test.env}

command="/usr/local/libexec/release_test/bin/release_test"
extra_commands="remote"
start_cmd="daemon -f -t $name -p $pidfile $command start"
remote_cmd=release_test_remote
#rpc_cmd=release_test_rpc

release_test_remote()
{
  export TERM=xterm
  $command remote
}

#release_test_rpc()
#{
#  $command rpc "$1"
#}

load_rc_config $name
run_rc_command "$@"
2 Likes