Multi-stage Docker image for phoenix

Hi,

I’m having trouble building a multi-stage Docker image for my app.

Here is my Dockerfile.base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

WORKDIR /elixir

COPY mix.* ./

RUN mix local.hex --force && \
  mix local.rebar --force && \
  mix deps.get && \
  mix deps.compile && \
  mix compile

I built and pushed it to the registry.
Then here is Dockerfile

FROM registry.gitlab.com/username/project_name/base:latest as base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

RUN apk --update --upgrade add inotify-tools bash postgresql-client \
    && update-ca-certificates --fresh \
    && rm -rf /var/cache/apk/*

WORKDIR /app

COPY . .
COPY --from=base /opt/mix /opt/mix
COPY --from=base /opt/hex /opt/hex
COPY --from=base /elixir/deps ./deps
COPY --from=base /elixir/_build ./_build

RUN mix deps.get \
    && mix deps.compile \
    && mix compile

It doesn’t fetch dependencies twice, this is good. But it tries to compile them again, and also fails at that.

Step 12/12 : RUN mix deps.get     && mix deps.compile     && mix compile
 ---> Running in 62fe07600112
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  certifi 2.5.1
  ...
  unicode_util_compat 0.4.1
All dependencies are up to date
===> Compiling parse_trans
===> Failed to restore /app/deps/parse_trans/.rebar3/erlcinfo file. Discarding it.

===> Compiling mimerl
===> Failed to restore /app/deps/mimerl/.rebar3/erlcinfo file. Discarding it.

...

What am I doing wrong? I thought it was enough to copy over the compiled deps from the base image.

Usually if you are doing a multi stage build it works best to just go ahead and build a release in the base image and then you just copy that into the runtime image. Then the runtime image can be way smaller.

3 Likes

This is for development. I’m not trying to build a release. The goal is that every team member can start the server or console as fast as possible.

I would suggest not going the multi-stage build route for local development. For releases it is definitely a good route to go down (and I also recently published a post on this recently here if you are interested: https://akoutmos.com/post/multipart-docker-and-elixir-1.9-releases/)

Given that your dependencies can/will change over time, maintaining a separate base image in my opinion will just lead to headaches across the team (people having out of date images, images needing to be updated every time there is a new dep/version, etc).

My current workflow (both for work and personal) is to have a docker-compose stack with my elixir service based on elixir:1.8.2 (or whatever version you are on), along with postgres and any other services i require. I then leverage named volumes (https://docs.docker.com/compose/compose-file/#short-syntax-3) so that every time I spin the container up, all the deps/build stuff is persisted from the last time the container ran.

This way everyone on the team has the same container versions, no need to introduce any additional CI/CD steps for deploying new images to the registry and what not. It is also fast and productive. I can usually have my whole stack up and running locally in under a minute.

2 Likes

Thanks, I will explore volumes for deps. Although my idea with multi stages was supposed to take care about deps and libs. It would compile them in base, but then still run mix deps.get and compile in the next stage when you start it. So if mix.exs hasn’t changed - nothing would be done. But if you added anything new, it would add and compile only that.

But regardless, now I’m just curious why copying deps from base image didn’t work.

Why even have a multistage build I guess is the question I have? Why not just base your container off of FROM registry.gitlab.com/username/project_name/base:latest? Seems like an extra step of indirection which doesn’t net you anything.

That‘s a good point. Honestly, I don‘t remember now, tried a couple of strategies today. I‘ll try it a bit later tonight and report back.

Personally I use them in Phoenix (for development at least) for Webpack. I have a stage in my Dockerfile that does nothing but install Node and build assets. Then in a 2nd stage that sets up Elixir / Phoenix / mix, etc. I copy in the compiled assets into a folder.

Then I run a dedicated webpack service in Docker Compose but only in development (using an override file).

Haven’t deployed anything yet but I imagine once I tackle that, I’ll end up moving the webpack stage to nginx and also set up a 2nd stage for Elixir releases to base as my final prod image.

That comment was aimed more at the OPs code sample where they used the registry image as a base, and then immediatly started the next stage:

FROM registry.gitlab.com/username/project_name/base:latest as base

FROM elixir:1.8-otp-22-alpine

For your case that sounds like a good solution as installing node+elixir in the same container can be annoying to do. When doing frontend work, I usually have 2 containers running, 1 for Elixir app and 1 for the JS app. Each container will then have read only access to the host filesystem (to the files that are relevant for each) and then kick off rebuilds as files change. At that point it is only a matter of setting up CORS correctly on the Phoenix app and my Vue SPA can communicate with my Phoenix API.

And because of that you make them install docker? Instead of just to teach them how to use mix? Perhaps asdf as well?

1 Like

That’s just how we like to develop, everything in docker containers. Nothing wrong with that.

1 Like

That’s how I did it as well. Webpack as a separate service. I actually used your repo as inspiration.

Ah cool. Things are much different now than that repo (using a single multi-stage build instead of 2x Dockerfiles). I was going to update it once I get deployment covered.

Oh wow, looking forward to the update. Could you maybe give a sneak preview on the Dockerfile for dev? Really need it right now :slight_smile:

Well every one to what they like, but for me it’s nothing I could get speed with… Constantly needing to rebuild the container slows me down.

Sure. Here’s the files I’m using now in a project I’m working on:

Dockerfile:

FROM node:10.15.3-stretch-slim as webpack
LABEL maintainer="Nick Janetakis <nick.janetakis@gmail.com>"

RUN npm install -g yarn

WORKDIR /app/assets

COPY assets/package.json assets/*yarn* ./

ENV BUILD_DEPS="build-essential" \
    APP_DEPS=""

RUN apt-get update \
  && apt-get install -y ${BUILD_DEPS} ${APP_DEPS} --no-install-recommends \
  && yarn install \
  && rm -rf /var/lib/apt/lists/* \
  && rm -rf /usr/share/doc && rm -rf /usr/share/man \
  && apt-get purge -y --auto-remove ${BUILD_DEPS} \
  && apt-get clean

COPY assets .

ARG NODE_ENV="production"
ENV NODE_ENV="${NODE_ENV}"

RUN if [ "${NODE_ENV}" != "development" ]; then \
  yarn run build; else mkdir -p /app/public; fi

CMD ["bash"]

#

FROM elixir:1.8.1-slim as app
LABEL maintainer="Nick Janetakis <nick.janetakis@gmail.com>"

WORKDIR /app

COPY mix* ./

ENV BUILD_DEPS="build-essential" \
    APP_DEPS="curl inotify-tools imagemagick"

RUN apt-get update \
  && apt-get install -y ${BUILD_DEPS} ${APP_DEPS} --no-install-recommends \
  && mix local.hex --force && mix local.rebar --force \
  && mix deps.get && mix deps.compile \
  && rm -rf /var/lib/apt/lists/* \
  && rm -rf /usr/share/doc && rm -rf /usr/share/man \
  && apt-get purge -y --auto-remove ${BUILD_DEPS} \
  && apt-get clean

ARG MIX_ENV="prod"
ENV MIX_ENV="${MIX_ENV}"

COPY --from=webpack /app/public /public

COPY . .

RUN chmod +x docker-entrypoint.sh
ENTRYPOINT ["/app/docker-entrypoint.sh"]

EXPOSE 8000

CMD ["mix", "phx.server"]

docker-compose.yml:

version: "3.4"

services:
  postgres:
    env_file:
      - ".env"
    image: "postgres:11.2"
    ports:
      - "127.0.0.1:5432:5432"
    restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
    stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
    volumes:
      - "postgres:/var/lib/postgresql/data"

  web:
    build:
      context: "."
      target: "app"
      args:
        - "MIX_ENV=${MIX_ENV:-prod}"
        - "NODE_ENV=${NODE_ENV:-production}"
    depends_on:
      - "postgres"
    env_file:
      - ".env"
    healthcheck:
      test: "${DOCKER_HEALTHCHECK_TEST:-curl localhost:8000/healthy}"
      interval: "60s"
      timeout: "3s"
      start_period: "5s"
      retries: 3
    image: "${DOCKER_WEB_IMAGE:-nickjj/docker-phoenix:latest}"
    ports:
      - "${DOCKER_WEB_PORT:-127.0.0.1:8000}:8000"
    restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
    stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
    tty: true
    volumes:
      - "${DOCKER_WEB_VOLUME:-./public:/app/priv/static}"

volumes:
  postgres: {}

docker-compose.override.yml:

version: "3.4"

services:
  webpack:
    build:
      context: "."
      target: "webpack"
      args:
        - "NODE_ENV=${NODE_ENV:-production}"
    command: "yarn run watch"
    env_file:
      - ".env"
    restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
    stop_grace_period: "${DOCKER_STOP_GRACE_PERIOD:-3s}"
    tty: true
    volumes:
      - ".:/app"

docker-entrypoint.sh:

#!/bin/bash

set -e

rm -rf /app/public/css /app/public/js /app/public/fonts /app/public/images

cp -a /public /app

exec "$@"

Couple things to note:

  • In mix.exs I have both build_path and deps_path set to /elixir/_build and /elixir/deps to avoid anything leaking out of the image back into my dev volume mounts.

  • I have a public/ folder in the root of the project that is the final end game destination of where assets will be so I can serve them with nginx. It ends up being a copy of what’s in priv/static.

  • Phoenix, Webpack and PostgreSQL all run in their own containers. Live reload works for assets but not for templates and Elixir code (limitation of Docker for Windows). However, changes take effect in about 1 second or less, so it’s a super fast development loop.

  • I have an .env file in the root of my project that populates all of the ENV variables you see in the Dockerfile and docker-compose.yml files.

1 Like

Thanks! I’ll try to get through it tomorrow.

This caused problems with my VSCode + ElixirLS extention. It could not find the deps location and highlighted the whole mix.exs file red. I tried to write the deps and _build to the original default location, but that causes problems apparently.

Weird. I tried VSCode, ElixirLS and remote containers and everything was found with the custom paths using that exact Docker set up. Autocomplete worked, linting worked, etc… It was pretty sweet.

It all worked out of the box mostly. I loosely followed this guide: https://ilhub.io/blog/2019/05/30/vscode-remote

Some stuff is out of date in that blog post (menu names) , but the general steps worked.

Yes, with remote containers. But I don’t use remote, because it required vscode insiders. Or it’s possible with normal vscode now?

When I used VSCode I only ever ran insiders. I’m not sure if it’s available in the stable release, but honestly, insiders was quite stable. I used it for like 6 months for hardcore every day development and it never crashed once.