Docker-compose phoenix postgresql

Phoenix Forum Questions / Help
, ,
1

I have a problem running my dockerized phoenix application with postgresql.

I specified my docker-compose.yml file to include two services: my phoenix service, build by a dockerfile and a latest postgres image taken from docker hub.

My dockerfile for phoenix looks like this:

# base image elixer to start with
FROM elixir:1.6

# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force

# install the latest phoenix 
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force

# create app folder
RUN mkdir /app
COPY ./my_app /app
WORKDIR /app

# install dependencies
RUN mix deps.get

# run phoenix in *dev* mode on port 4000
# CMD mix phx.server

My docker-compose.yml file looks like this:

version: '3.6'
services:
  phoenix:
    build:
      context: .
      dockerfile: Dockerfile.phoenix.development
    ports:
      - 4000:4000
    volumes:
      - ./my_app:/app
    depends_on:
      - db
      - redis
    environment:
      GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
      GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET}
      FACEBOOK_CLIENT_ID: ${FACEBOOK_CLIENT_ID}
      FACEBOOK_CLIENT_SECRET: ${FACEBOOK_CLIENT_SECRET}
    command: mix ecto.create && mix ecto.migrate && mix phx.server
  db:
    container_name: db
    restart: always
    image: postgres:latest
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_PASSWORD="some_pass"
      - POSTGRES_USER="some_user"
    volumes:
          - ./data/db:/data/db

Here is the error:

phoenix_1  | 
phoenix_1  | 10:07:19.148 [error] GenServer #PID<0.234.0> terminating
phoenix_1  | ** (RuntimeError) Connect raised a KeyError error. The exception details are hidden, as
phoenix_1  | they may contain sensitive data such as database credentials.
phoenix_1  | 
phoenix_1  |     (elixir) lib/keyword.ex:377: Keyword.fetch!/2
phoenix_1  |     (postgrex) lib/postgrex/protocol.ex:577: Postgrex.Protocol.startup/2
phoenix_1  |     (postgrex) lib/postgrex/protocol.ex:504: Postgrex.Protocol.handshake/2
phoenix_1  |     (db_connection) lib/db_connection/connection.ex:135: DBConnection.Connection.connect/2
phoenix_1  |     (connection) lib/connection.ex:622: Connection.enter_connect/5
phoenix_1  |     (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
phoenix_1  | Last message: nil
phoenix_1  | State: Postgrex.Protocol
db         | 2018-07-24 10:07:19.149 UTC [28] LOG:  incomplete startup packet
phoenix_1  | ** (Mix) The database for MyApp.Repo couldn't be created: an exception was raised:
phoenix_1  |     ** (RuntimeError) Connect raised a KeyError error. The exception details are hidden, as
phoenix_1  | they may contain sensitive data such as database credentials.
phoenix_1  | 
phoenix_1  |         (elixir) lib/keyword.ex:377: Keyword.fetch!/2
phoenix_1  |         (postgrex) lib/postgrex/protocol.ex:577: Postgrex.Protocol.startup/2
phoenix_1  |         (postgrex) lib/postgrex/protocol.ex:504: Postgrex.Protocol.handshake/2
phoenix_1  |         (db_connection) lib/db_connection/connection.ex:135: DBConnection.Connection.connect/2
phoenix_1  |         (connection) lib/connection.ex:622: Connection.enter_connect/5
phoenix_1  |         (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

I’m able to run my phoenix app on localhost just fine without docker.

When I set this up with docker and docker-compose, I remember to switch the dev.exs file to reference the host “db”.

I wondering if this is an issue with my postgresql setup or my phoenix app setup

2

So the DB receives something.

Can you please check your exact versions of the database and the postgrex package used?

Also how does your configuration for the database looks like? Host, Port, Password, Username, etc?

3

:postgrex, “>= 0.0.0”,

host, port, password, username are correct from my understanding

4

I asked for the exact version from your mix.lock as it is in the Dockercontainer.

Maybe our or elixirs understand differs from yours, could you just show them? Replacing literal secrets by dummy-values of course.

5

“postgrex”: {:hex, :postgrex, “0.13.5”

config :my_app, MyApp.Repo,
adapter: Ecto.Adapters.Postgres,
username: System.get_env(“POSTGRES_USER”),
password: System.get_env(“POSTGRES_PASSWORD”),
database: “my_postgres_dev”,
hostname: “db”,
port: 5432,
pool_size: 10

maybe docker postgresql:latest doesn’t match postgrex?

6

Where does your phoenix container know those values from? You are only setting them in db, so System.get_env/1 will return nil, which means in this context, that it shall delete the associated key.

Also, could you please markdown to properly format your code?

7

Sorry my bad.

The system should know this from the environment specified in the docker-compose.yml file under the service db that uses the latest postgres image since i hardcoded them there

Should i set them in the phoenix service?

8

TL;DR: Set them in a way, that makes you able to keep them in synch easily.

The environment key is always for the surrounding image only.

Usually you are passing those environment variables to the postgresql container to configure it.

Then you have to makje sure that all clients use the same credentials as passed into the postgres container. How you do this often depends on your overall architecture.

Most of the time its just hard-configuring them in the config/<env>.exs or passing them as environment variable into the client container as well.

But vault managed credentials are possible as well. Well, kind of… Changing the credentials for a repo requires it to kill the repos supervisor and restart it with new credentials as far as I understand…

2 Likes
9

Each service needs its own reference to environment variables. You’ll want to add it to the phoenix service too.

You could also put those ENV variables into a file, such as a .env file and then load it in both services with env_file in compose (instead of inlining and duplicating them with environment).

2 Likes
10

Thanks @cnck1387 I set my POSTGRES_USER and POSTGRES_PASSWORD in the .env file, but it does not seem to work. The error I get back is password authentication failed for user “my_username”. I use this same username and password on my localhost without containers and it works just fine. Just not with containers

11

That’s because the postgres image creates a default database and database user based on those environment variables on the first time it runs (assuming a volume is persisted).

Perhaps you changed those values after you’ve upped your project?

If that’s the case you’ll want to delete your postgres volume which you could do by running docker-compose down -v, but please don’t run that in production. That’s just meant as a quick and easy way to destroy all volumes which is handy for development.

If you’re in production you would want to either change your user and password values to match what they were when you first ran your compose project, or use psql to manually change the user / password to whatever new ones you set (similar to what you would do without Docker).

12

Yes this did the trick, I also had to add this to my dockerfile

CMD mix ecto.create && mix ecto.migrate

before doing

CMD mix phx.server

Do I need to be concerned with this log?

db         | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db         | 
db         | waiting for server to shut down....2018-07-24 16:55:43.497 UTC [38] LOG:  received fast shutdown request
db         | 2018-07-24 16:55:43.499 UTC [38] LOG:  aborting any active transactions
db         | 2018-07-24 16:55:43.508 UTC [38] LOG:  worker process: logical replication launcher (PID 45) exited with exit code 1
db         | 2018-07-24 16:55:43.509 UTC [40] LOG:  shutting down
db         | 2018-07-24 16:55:43.530 UTC [38] LOG:  database system is shut down
db         |  done
db         | server stopped
db         | 
db         | PostgreSQL init process complete; ready for start up.
db         | 
db         | 2018-07-24 16:55:43.609 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db         | 2018-07-24 16:55:43.609 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db         | 2018-07-24 16:55:43.613 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db         | 2018-07-24 16:55:43.629 UTC [65] LOG:  database system was shut down at 2018-07-24 16:55:43 UTC
db         | 2018-07-24 16:55:43.635 UTC [1] LOG:  database system is ready to accept connections
13

I wouldn’t put create / migrate in your Dockerfile. Those are really something you do at runtime, not build time.

I would just docker-compose exec those commands after you up everything beforehand. Or if you were really gung-ho on having that done automatically on every container start you could put it into an entrypoint script but personally I don’t like doing that.

14

so should I do something like:

ENTRYPOINT mix ecto.create && mix ecto.migrate

# then
CMD mix phx.server

And what about the “error/warning” logs I received in my above comment, should I be concerned about those? Does this mean that postgres is not started up properly?

15

As far as I remember, that’s invalid syntax. Is just create a shell script and use that as an entry point.

17

What I understand here is that I should be running mix ecto.create and mix ecto.migrate before running mix phx.server. If I put these as entrypoints inside of a shell script and run this, my docker container crashes since mix phx.server depends on both ecto create and migrate commands

18

I do not understand that one…

ENTRYPOINT /usr/bin/entrypoint.sh

#!/usr/bin/env bash
set -ex

mix ecto.create
mix ecto.migrate

mix phx.server
1 Like
19

Also, to make sure that your database has been created with the correct credentials, please purge your volume.

Also for development you really shouldn’t use hard volumes as you did. That could even interfere with a real PostgreSQL in your development machine.

20

when you say purge my volume, do you mean docker-compose down -v?

what do you mean by a hard volume?

21

It doesn’t crash for me with the example in the github repo I linked.

But you could always do docker-compose run phoenix mix ecto.create && mix ecto.migrate before you do docker-compose up with your example.