Adding Solid Docker Compose Recipes

The shadowing issue is only an issue if you run mix deps.get and mix compile in both the host and the container, otherwise is a non issue. If you run in both than you just need to map each folder in your root project and leave out the deps, _build and I think the assets/node_modules folders.

This is a completely different issue, and is nothing related with shadowing, instead it means you are missing the OS dependencies to build bccrypt in the container or if you are running mix commands in both the host and in the container, then it may cause issues, because bcrypt was compiled for a different target, but without seeing the exact error is hard to say.

To resume if you don’t run mix commands in both the container and host, shadowing is a non issue.

Docker Stack

Dockerfile

The Dockerfile now as non root user with the id 1000. Please read the comments in the Dockerfile to see if you need to customize this id for your computer.

ARG TAG=latest

FROM elixir:${TAG}

ARG PHOENIX_VERSION=1.4.16

RUN apt-get update && \

  ## install inotify-tool for phoenix hot reload
  apt-get install -y inotify-tools && \
  # DO THE SAME IN YOUR HOST MACHINE
  printf "fs.inotify.max_user_watches=524288\n" > /etc/sysctl.d/01-inotify.conf && \

  # install nodejs and npm on globaly
  curl -sL https://deb.nodesource.com/setup_14.x | bash - && \
  apt-get install -y -q nodejs && \

  # Add the user `developer` with id `1000`. Adjust `1000` to match the user in
  #  your host in order to avoid permissions issues between container and host.
  # To check user id in host, just run: `id -u`
  useradd -m -u 1000 -s /bin/bash developer && \
  su developer -c 'mkdir -p /home/developer/phoenix' && \

  # Install postgres client
  apt-get install -y postgresql-client && \
  # Clear apt cache
  rm -rf /var/lib/apt/lists/*

USER developer

RUN mix local.hex --force &&\
    mix local.rebar --force &&\
    mix archive.install hex phx_new ${PHOENIX_VERSION} --force

WORKDIR /home/developer/phoenix

CMD ["bash"]

You can now build any combination of Elixir and Phoenix version:

docker build --build-arg "TAG=1.10" --build-arg "PHOENIX_VERSION=1.5.1" -t phoenix:1.10_1.5.1 .

Docker Compose

I don’t have Elixir installed in my host, therefore I cannot test this docker compose file, but if I have not missed anything it will allow you to run mix commands from the host and container without causing issues:

# DONT USER VERSION £ UNLESS YOUR ARE USING DOCKER SWARM TO RUN THE CONTIANERS
version: '2.3'

services:
  db:
    image: postgres:9.6
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      PGDATA: /var/lib/postgresql/data/pgdata
    restart: always
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    ports:
      # IF POSTGRES IS ONLY USED BY THE PHOENIX SERVICE THEN YOU CAN REMOVE THIS PORT MAP
      # This is listening to requests coming from outside your computer:
      #- "5432:5432"
      # Localhost only
      - "127.0.0.1:5432:5432"
  phoenix:
    build: .
    volumes:
      - $PWD/assets/css:/home/developer/phoenix/assets/css
      - $PWD/assets/js:/home/developer/phoenix/assets/js
      - $PWD/assets/static:/home/developer/phoenix/assets/static
      - $PWD/assets/package.json:/home/developer/phoenix/assets/package.json
      - $PWD/assets/package-lock.json:/home/developer/phoenix/assets/package-lock.json
      - $PWD/assets/webpack.config.js:/home/developer/phoenix/assets/webpack.config.js
      - $PWD:/home/developer/phoenix/config
      - $PWD:/home/developer/phoenix/lib
      - $PWD:/home/developer/phoenix/priv
      - $PWD/mix.exs:/home/developer/phoenix/mix.exs
      - $PWD/mix.lock:/home/developer/phoenix/mix.lock
      - $PWD/.formatter.exs:/home/developer/phoenix/.formatter.exs
    
    # DONT KNOW WHAT IS DOING BUT PROBABLY YOU WANT TO ADJUST IT FOR THE NEW STACK.
    #command: ../phoenix-setup.sh
    command: mix phx.server
    environment:
      PGUSER: postgres
      PGPASSWORD: postgres
      PGDATABASE: portfolio
      PGPORT: 5432
      PGHOST: db
    ports:
      # Only visible to localhost, otherwise will be 0.0.0.0, meaning it will listen to requests coming from outsiide.
      - "127.0.0.1:4000:4000"
    depends_on:
      - db
volumes:
  pgdata:

Well this is a nice fix but I don’t think that it resolve the shadowing issues of deps folder in the container, by using volumes it will pull all the local files into the container and shadow the stuff in the container… I just tried it and I got an error while loading Bcrypt on a sign-in method for example.

So just remove from your Dockerfile:

COPY . /phoenix

EXPOSE 4000

Add this:

WORKDIR /app

and fix your docker compose volume to:

- $PWD:/app

But his is running as root, and that is not good from a security point of view.

With this changes you don’t need to rebuild your image each time you change files in the host.

Just add inotify tools to the container and host OS and change the max number of files he can watch, and you are good to go.

#!/bin/sh

set -eu

Main()
{
    apt install -y -q --no-install-recommends \
        procps \
        inotify-tools

    printf "fs.inotify.max_user_watches=524288\n" > /etc/sysctl.d/01-inotify.conf
}

Main

I use this for years without any problems.

I am working on an Elixir Docker Stack that just makes everything a breeze, but not yet ready for others to try because the base image is not yet released in Docker Hub, therefore you cannot try it in your computer.

Generaly I just use my terminal not the one from the editor. For now i’m not working with docker containers on dev env as I need to restart them on each changes.

Just a quick question… Do you run your project code from within your editor, be it for running tests or anything else, like to open a iex session?

With pleasure !

Here is the new project directory:

Screenshot 2020-05-12 at 15.31.10504×542 19.6 KB

The Dockerfile :

FROM elixir:latest


RUN mix local.hex --force &&\
    mix local.rebar --force &&\
    #This part could be removed as we do not need to install phoenix dependencies.
    mix archive.install hex phx_new 1.4.16 --force

RUN apt-get update && \
  ## install inotify-tool for phoenix hot reload
  apt-get install -y inotify-tools && \
  # install nodejs and npm on globaly
  curl -sL https://deb.nodesource.com/setup_14.x | bash - && \
   apt-get install -y -q nodejs && \
  # Install postgres client
  apt-get install -y postgresql-client && \
  # Clear apt cache
   rm -rf /var/lib/apt/lists/*

WORKDIR /phoenix

COPY . /phoenix

EXPOSE 4000

The docker-compose file:

version: '3.7'

services:
  db:
    image: postgres:9.6
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      PGDATA: /var/lib/postgresql/data/pgdata
    restart: always
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    ports:
      - "5432:5432"
  phoenix:
    build: .
    volumes:
      - .docker/phoenix/phoenix-setup.sh:/phoenix-setup.sh
    command: ../phoenix-setup.sh
    environment:
      PGUSER: postgres
      PGPASSWORD: postgres
      PGDATABASE: portfolio
      PGPORT: 5432
      PGHOST: db
    ports:
      - "4000:4000"
    depends_on:
      - db
volumes:
  pgdata:

And .dockerignore:

deps
_build
assets/node_modules
test

If you have any suggestion/reviews I will love to hear them.

Thanks a lot !

Share your Dockerfile and docker compose so that we can try to help you, because I use Docker for all my development stuff, including to run the Android Studio editor, that’s a very complex and heavy tool, and things just works, provided you have the correct setup and Operating System

I use an Elixir Docker Stack with a lot of bash helpers that makes Elixir development a breeze and a joy, and I also run the editor from inside another docker container.

Ok guys so I found that when you use volumes in docker-compose your create some side effects that could result in problems.

When you use the volume and pull everything from your project root directory you are overloading the content of the compiled stuff in your container so some dependencies won’t be “accessible” as their dependencies need to be compiled in order to work correctly.

Off course correct me if i’m wrong.

From my initial post I had a .docker folder that contains the necessary Dockerfile for my image with a setup shell script and in my project root directory I had my docker-compose file that was pulling all the project and target the project folder in my container.

Now I moved out my dockerfile in my project root directory because I canot copy files from a parent path of the context and even if it’s logic I do not really get how to keep my dockerfile in the docker folder, this is not much but I was usually using volumes in my compose files also to pull the source code even after a change. I’m also a little bit confused because this mean that at every change you need to rebuild the project, that’s a time waste.

At least you could only copy your mix.deps files and compile them but if you’re still using volumes you will override the deps and _build folder. For now I’m staying with this approach but do not hesitate to share your reviews !

The almighty @NobbZ explained very well all the side effects that this would cause and how a container should really behave: here.

Thanks a lot man !

I leverage Docker Compose for most of my tutorials on my blog. Below is a listing of the tech stacks, articles, and repos for each of the tutorials:

Prometheus, PostGIS and Phoenix Part 1
Article: Prometheus, PostGIS and Phoenix Part 1-Alex Koutmos | Engineering Blog
Repo: GitHub - akoutmos/elixir_monitoring_prom: Companion code for https://akoutmos.com/post/prometheus-postgis-and-phoenix/ · GitHub
Tech Stack: Elixir app, Postgres+PostGIS, Grafana, Prometheus, Postgres Exporter

Broadway, RabbitMQ, and the Rise of Elixir Part 1
Article: Broadway, RabbitMQ, and the Rise of Elixir Part 1-Alex Koutmos | Engineering Blog
Repo: GitHub - akoutmos/elixir_popularity: Companion code for https://akoutmos.com/post/broadway-rabbitmq-and-the-rise-of-elixir/ · GitHub
Tech Stack: cAdvisor, RabbitMQ, Postgres, Postgres Exporter, Grafana, Prometheus

Structured logging in Elixir using Loki
Article: Structured logging in Elixir using Loki-Alex Koutmos | Engineering Blog
Repo: GitHub - akoutmos/auto_finder: Companion code for https://akoutmos.com/post/elixir-logging-loki/ · GitHub
Tech Stack: Elixir app, Postgres, Loki, Grafana

And finally, while not Docker Compose specific, how to build Mix Releases with Docker:

Multi-stage Docker Builds and Elixir 1.9 Releases
Article: Multi-stage Docker Builds and Elixir 1.9 Releases-Alex Koutmos | Engineering Blog
Repo: GitHub - akoutmos/docker_elixir_19_release: Companion code for https://akoutmos.com/post/multipart-docker-and-elixir-1.9-releases/ · GitHub

Sweet Work.