egze
Multi-stage Docker image for phoenix
Hi,
I’m having trouble building a multi-stage Docker image for my app.
Here is my Dockerfile.base
FROM elixir:1.8-otp-22-alpine
ENV MIX_HOME=/opt/mix \
HEX_HOME=/opt/hex
WORKDIR /elixir
COPY mix.* ./
RUN mix local.hex --force && \
mix local.rebar --force && \
mix deps.get && \
mix deps.compile && \
mix compile
I built and pushed it to the registry.
Then here is Dockerfile
FROM registry.gitlab.com/username/project_name/base:latest as base
FROM elixir:1.8-otp-22-alpine
ENV MIX_HOME=/opt/mix \
HEX_HOME=/opt/hex
RUN apk --update --upgrade add inotify-tools bash postgresql-client \
&& update-ca-certificates --fresh \
&& rm -rf /var/cache/apk/*
WORKDIR /app
COPY . .
COPY --from=base /opt/mix /opt/mix
COPY --from=base /opt/hex /opt/hex
COPY --from=base /elixir/deps ./deps
COPY --from=base /elixir/_build ./_build
RUN mix deps.get \
&& mix deps.compile \
&& mix compile
It doesn’t fetch dependencies twice, this is good. But it tries to compile them again, and also fails at that.
Step 12/12 : RUN mix deps.get && mix deps.compile && mix compile
---> Running in 62fe07600112
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
certifi 2.5.1
...
unicode_util_compat 0.4.1
All dependencies are up to date
===> Compiling parse_trans
===> Failed to restore /app/deps/parse_trans/.rebar3/erlcinfo file. Discarding it.
===> Compiling mimerl
===> Failed to restore /app/deps/mimerl/.rebar3/erlcinfo file. Discarding it.
...
What am I doing wrong? I thought it was enough to copy over the compiled deps from the base image.
Most Liked
akoutmos
I would suggest not going the multi-stage build route for local development. For releases it is definitely a good route to go down (and I also recently published a post on this recently here if you are interested: Multi-stage Docker Builds and Elixir 1.9 Releases-Alex Koutmos | Engineering Blog)
Given that your dependencies can/will change over time, maintaining a separate base image in my opinion will just lead to headaches across the team (people having out of date images, images needing to be updated every time there is a new dep/version, etc).
My current workflow (both for work and personal) is to have a docker-compose stack with my elixir service based on elixir:1.8.2 (or whatever version you are on), along with postgres and any other services i require. I then leverage named volumes (Compose file reference | Docker Docs) so that every time I spin the container up, all the deps/build stuff is persisted from the last time the container ran.
This way everyone on the team has the same container versions, no need to introduce any additional CI/CD steps for deploying new images to the registry and what not. It is also fast and productive. I can usually have my whole stack up and running locally in under a minute.
egze
If anyone is interested, here is my latest Docker setup. I hardcoded everything for development. For production I will probably do it differently and build a release. But for development it works just great.
docker/dev/Dockerfile
FROM node:12.4-alpine as webpack
RUN npm install -g yarn
WORKDIR /app/assets
COPY assets/package.json assets/*yarn* ./
RUN apk --update --upgrade add ca-certificates build-base bash \
&& update-ca-certificates --fresh \
&& rm -rf /var/cache/apk/* \
&& yarn install \
&& apk del build-base
COPY assets .
ENV NODE_ENV="development"
CMD ["tail", "-f", "/dev/null"]
###############################
FROM elixir:1.8-otp-22-alpine as app
WORKDIR /app
RUN apk --update --upgrade add ca-certificates inotify-tools postgresql-client bash \
&& update-ca-certificates --fresh \
&& rm -rf /var/cache/apk/*
COPY ./docker/dev/wait-for-postgres.sh /
COPY mix.* ./
ENV MIX_ENV="dev" \
MIX_HOME=/opt/mix \
HEX_HOME=/opt/hex
RUN mix local.hex --force \
&& mix local.rebar --force \
&& mix deps.get \
&& mix deps.compile
CMD ["tail", "-f", "/dev/null"]
docker-compose.yml
version: '3.7'
services:
postgres:
image: postgres:11-alpine
container_name: project_postgres
restart: always
ports:
- 15432:5432
volumes:
- ./docker/data/postgres:/var/lib/postgresql/data
webpack:
build:
context: .
dockerfile: ./docker/dev/Dockerfile
target: "webpack"
container_name: project_webpack
restart: unless-stopped
command: yarn run watch
volumes:
- .:/app
- static:/app/priv/static
app:
build:
context: .
dockerfile: ./docker/dev/Dockerfile
target: "app"
container_name: project_app
ports:
- 4000:4000
command: tail -f /dev/null
volumes:
- .:/app
- static:/app/priv/static
- deps:/app/deps
- build:/app/_build
depends_on:
- postgres
- webpack
volumes:
static: {}
deps: {}
build: {}
docker/dev/wait-for-postgres.sh
#!/bin/bash
# wait-for-postgres.sh
set -e
host="$1"
shift
cmd="$@"
until PGPASSWORD=$POSTGRES_PASSWORD psql -h "$host" -U "postgres" -c '\q'; do
>&2 echo "Postgres is unavailable - sleeping"
sleep 1
done
>&2 echo "Postgres is up - executing command"
exec $cmd
And Makefile
.PHONY: app server console sh clean
app:
docker-compose up --detach --build app
setup: app
docker-compose exec app /wait-for-postgres.sh postgres mix ecto.setup
server: setup
docker-compose exec app mix phx.server
console: setup
docker-compose exec app iex -S mix
sh: setup
docker-compose exec app bash
clean:
docker-compose down
Now I can just execute make console or make server and it will start all services and even wait for postgres to be healthy and execute migrations before starting the server on iex.
Thanks @cnck1387 for the helpful tips.
benwilson512
Usually if you are doing a multi stage build it works best to just go ahead and build a release in the base image and then you just copy that into the runtime image. Then the runtime image can be way smaller.
Last Post!
tme_317
Hi @cnck1387,
I was wondering if you have these dockerfiles and compose files open sourced? I’ve read many blog posts with slightly different approaches and just not sure how to tie it all together for a basic generated phoenix umbrella app (with ecto). I’m trying to get a development environment (editing using vscode remote container plugin) and a mix release driven multi-stage build delivering the .tar.gz release ready for deployment in a container in the same project.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









