egze

egze

Multi-stage Docker image for phoenix

Hi,

I’m having trouble building a multi-stage Docker image for my app.

Here is my Dockerfile.base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

WORKDIR /elixir

COPY mix.* ./

RUN mix local.hex --force && \
  mix local.rebar --force && \
  mix deps.get && \
  mix deps.compile && \
  mix compile

I built and pushed it to the registry.
Then here is Dockerfile

FROM registry.gitlab.com/username/project_name/base:latest as base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

RUN apk --update --upgrade add inotify-tools bash postgresql-client \
    && update-ca-certificates --fresh \
    && rm -rf /var/cache/apk/*

WORKDIR /app

COPY . .
COPY --from=base /opt/mix /opt/mix
COPY --from=base /opt/hex /opt/hex
COPY --from=base /elixir/deps ./deps
COPY --from=base /elixir/_build ./_build

RUN mix deps.get \
    && mix deps.compile \
    && mix compile

It doesn’t fetch dependencies twice, this is good. But it tries to compile them again, and also fails at that.

Step 12/12 : RUN mix deps.get     && mix deps.compile     && mix compile
 ---> Running in 62fe07600112
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  certifi 2.5.1
  ...
  unicode_util_compat 0.4.1
All dependencies are up to date
===> Compiling parse_trans
===> Failed to restore /app/deps/parse_trans/.rebar3/erlcinfo file. Discarding it.

===> Compiling mimerl
===> Failed to restore /app/deps/mimerl/.rebar3/erlcinfo file. Discarding it.

...

What am I doing wrong? I thought it was enough to copy over the compiled deps from the base image.

Most Liked

akoutmos

akoutmos

Author of Build a Weather Station with Elixir and Nerves

I would suggest not going the multi-stage build route for local development. For releases it is definitely a good route to go down (and I also recently published a post on this recently here if you are interested: Multi-stage Docker Builds and Elixir 1.9 Releases-Alex Koutmos | Engineering Blog)

Given that your dependencies can/will change over time, maintaining a separate base image in my opinion will just lead to headaches across the team (people having out of date images, images needing to be updated every time there is a new dep/version, etc).

My current workflow (both for work and personal) is to have a docker-compose stack with my elixir service based on elixir:1.8.2 (or whatever version you are on), along with postgres and any other services i require. I then leverage named volumes (Compose file reference | Docker Docs) so that every time I spin the container up, all the deps/build stuff is persisted from the last time the container ran.

This way everyone on the team has the same container versions, no need to introduce any additional CI/CD steps for deploying new images to the registry and what not. It is also fast and productive. I can usually have my whole stack up and running locally in under a minute.

egze

egze

If anyone is interested, here is my latest Docker setup. I hardcoded everything for development. For production I will probably do it differently and build a release. But for development it works just great.

docker/dev/Dockerfile

FROM node:12.4-alpine as webpack

RUN npm install -g yarn

WORKDIR /app/assets

COPY assets/package.json assets/*yarn* ./

RUN apk --update --upgrade add ca-certificates build-base bash \
    && update-ca-certificates --fresh \
    && rm -rf /var/cache/apk/*  \
    && yarn install \
    && apk del build-base

COPY assets .

ENV NODE_ENV="development"

CMD ["tail", "-f", "/dev/null"]

###############################

FROM elixir:1.8-otp-22-alpine as app

WORKDIR /app

RUN apk --update --upgrade add ca-certificates inotify-tools postgresql-client bash \
    && update-ca-certificates --fresh \
    && rm -rf /var/cache/apk/*

COPY ./docker/dev/wait-for-postgres.sh /
COPY mix.* ./

ENV MIX_ENV="dev" \
    MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

RUN mix local.hex --force \
    && mix local.rebar --force \
    && mix deps.get \
    && mix deps.compile

CMD ["tail", "-f", "/dev/null"]

docker-compose.yml

version: '3.7'

services:
  postgres:
    image: postgres:11-alpine
    container_name: project_postgres
    restart: always
    ports:
      - 15432:5432
    volumes:
      - ./docker/data/postgres:/var/lib/postgresql/data

  webpack:
    build:
      context: .
      dockerfile: ./docker/dev/Dockerfile
      target: "webpack"
    container_name: project_webpack
    restart: unless-stopped
    command: yarn run watch
    volumes:
      - .:/app
      - static:/app/priv/static

  app:
    build:
      context: .
      dockerfile: ./docker/dev/Dockerfile
      target: "app"
    container_name: project_app
    ports:
      - 4000:4000
    command: tail -f /dev/null
    volumes:
      - .:/app
      - static:/app/priv/static
      - deps:/app/deps
      - build:/app/_build
    depends_on:
      - postgres
      - webpack

volumes:
  static: {}
  deps: {}
  build: {}

docker/dev/wait-for-postgres.sh

#!/bin/bash
# wait-for-postgres.sh

set -e

host="$1"
shift
cmd="$@"

until PGPASSWORD=$POSTGRES_PASSWORD psql -h "$host" -U "postgres" -c '\q'; do
  >&2 echo "Postgres is unavailable - sleeping"
  sleep 1
done

>&2 echo "Postgres is up - executing command"
exec $cmd

And Makefile

.PHONY: app server console sh clean

app:
	docker-compose up --detach --build app

setup: app
	docker-compose exec app /wait-for-postgres.sh postgres mix ecto.setup

server: setup
	docker-compose exec app mix phx.server

console: setup
	docker-compose exec app iex -S mix

sh: setup
	docker-compose exec app bash

clean:
	docker-compose down

Now I can just execute make console or make server and it will start all services and even wait for postgres to be healthy and execute migrations before starting the server on iex.

Thanks @cnck1387 for the helpful tips.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Usually if you are doing a multi stage build it works best to just go ahead and build a release in the base image and then you just copy that into the runtime image. Then the runtime image can be way smaller.

Last Post!

tme_317

tme_317

Hi @cnck1387,

I was wondering if you have these dockerfiles and compose files open sourced? I’ve read many blog posts with slightly different approaches and just not sure how to tie it all together for a basic generated phoenix umbrella app (with ecto). I’m trying to get a development environment (editing using vscode remote container plugin) and a mix release driven multi-stage build delivering the .tar.gz release ready for deployment in a container in the same project.

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36820 110
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement