Load env vars with docker-compose env_file, secrets

Docker secrets only work when you are using docker swarm.

And the environment variables should be available but perhaps you are reading them at the wrong time?

Can you tell us more about your build process and how you read from the environment?

I haven’t deployed my app yet, but in development environment variables are working with Docker Compose using the same patterns that worked with other applications.

I have a .env file in the same directory as my docker-compose.yml file and then in the compose file I use:

    env_file:
      - ".env"

At this point all of the env variables are available in my Phoenix config files. For example you can use System.get_env("SOME_ENV_VAR_FROM_THE_ENV_FILE").

Are you doing something different, or maybe you forgot the env_file property on one of your services?

Is env_file: a child of services: level or a child of my_app: level?

I’m open to trying anything at this point.

It’s a child of myapp. Each individual service can have its own list of environment files (they are loaded and merged from top to bottom since technically you can have more than 1 file).

Basically, Docker isn’t going to automatically read in your environment variables. By setting env_file on a specific service, that instructs Docker to make them available.

The only file that gets automatically used is a .env file, but that’s for setting things like the COMPOSE_PROJECT_NAME or using variable substitution in the docker-compose.yml file itself. The vars in the .env file won’t get loaded into your services unless you explicitly set it in env_file.

Edit: There’s also the environment property which you can use either in combination with or as an alternative to env_file. The environment property lets you pass in key / value pairs of environment variables instead of having them loaded in from a file. Could be useful in some cases.

@maz, let me ask you again, how exactly is your build process?

The Dockerfile won’t see environment variables that are only injected at container runtime via an env file. So depending on if you are doing compiletime or runtime configuration, the environment might already be backed in as “empty”.

Sorry about the lack of response, I only had time this morning to make a quick response to @cnck1387. This is my docker compose with the env changes(they are not yet checked-in from my linux box at home which is where I can do quick turnaround development with docker)

I set - port: to localhost:4000:4000, commented out the web network, and commented-out the traefik directives in order to speed up iterations:

for building I do:

docker-compose pull 
docker-compose build --pull olivetree
docker-compose up --build -d postgres
docker-compose run --rm olivetree seed
docker-compose run --rm olivetree generate_hash_ids
docker-compose up --build olivetree

So going from what you are saying, I’d need access to those environment variables at least by docker-compose build --pull olivetree but that doesn’t seem to be the case.

[EDIT] added Dockerfile to gist.

yeah I’m still not sure why my env_file:\n - ./olivetree.env is not being honored. Or maybe it is but the timing of the SET is too late. It might be something I’m doing/not doing in my build process.

You can’t access environment variables from env_file at build time. It’s for run time.

If you want env variables to be set at build time, take a look at build arguments in the Docker Compose documentaton.

Right, so I guess I would want, say, System.get_env("GUARDIAN_SECRET_KEY") in config.exs to resolve at buildtime? Is there any situation where I’d prefer get_env() to resolve at runtime? Because if they can only be resolved at buildtime, I am barking up the wrong tree.

I never messed around with releases, but in development using mix to launch Phoenix, System.get_env("FOO") will resolve at run time and it will obtain that value thanks to Docker Compose making that environment variable available to Phoenix due to env_file (or environment) in the docker-compose.yml file.

You can double check what’s available in your container by running docker-compose run olivetree env. It should return back all of your environment variables that you set.