Multi-stage Docker image for phoenix

egze

Hi,

I’m having trouble building a multi-stage Docker image for my app.

Here is my Dockerfile.base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

WORKDIR /elixir

COPY mix.* ./

RUN mix local.hex --force && \
  mix local.rebar --force && \
  mix deps.get && \
  mix deps.compile && \
  mix compile

I built and pushed it to the registry.
Then here is Dockerfile

FROM registry.gitlab.com/username/project_name/base:latest as base

FROM elixir:1.8-otp-22-alpine

ENV MIX_HOME=/opt/mix \
    HEX_HOME=/opt/hex

RUN apk --update --upgrade add inotify-tools bash postgresql-client \
    && update-ca-certificates --fresh \
    && rm -rf /var/cache/apk/*

WORKDIR /app

COPY . .
COPY --from=base /opt/mix /opt/mix
COPY --from=base /opt/hex /opt/hex
COPY --from=base /elixir/deps ./deps
COPY --from=base /elixir/_build ./_build

RUN mix deps.get \
    && mix deps.compile \
    && mix compile

It doesn’t fetch dependencies twice, this is good. But it tries to compile them again, and also fails at that.

Step 12/12 : RUN mix deps.get     && mix deps.compile     && mix compile
 ---> Running in 62fe07600112
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  certifi 2.5.1
  ...
  unicode_util_compat 0.4.1
All dependencies are up to date
===> Compiling parse_trans
===> Failed to restore /app/deps/parse_trans/.rebar3/erlcinfo file. Discarding it.

===> Compiling mimerl
===> Failed to restore /app/deps/mimerl/.rebar3/erlcinfo file. Discarding it.

...

What am I doing wrong? I thought it was enough to copy over the compiled deps from the base image.

27 comments

/phoenix #docker #dependencies

1 6581 27

2020-02-01 13:55:51 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Usually if you are doing a multi stage build it works best to just go ahead and build a release in the base image and then you just copy that into the runtime image. Then the runtime image can be way smaller.

Post #1

egze

This is for development. I’m not trying to build a release. The goal is that every team member can start the server or console as fast as possible.

Post #2

akoutmos

Author of Build a Weather Station with Elixir and Nerves

I would suggest not going the multi-stage build route for local development. For releases it is definitely a good route to go down (and I also recently published a post on this recently here if you are interested: Multi-stage Docker Builds and Elixir 1.9 Releases-Alex Koutmos | Engineering Blog)

Given that your dependencies can/will change over time, maintaining a separate base image in my opinion will just lead to headaches across the team (people having out of date images, images needing to be updated every time there is a new dep/version, etc).

My current workflow (both for work and personal) is to have a docker-compose stack with my elixir service based on elixir:1.8.2 (or whatever version you are on), along with postgres and any other services i require. I then leverage named volumes (Compose file reference | Docker Docs) so that every time I spin the container up, all the deps/build stuff is persisted from the last time the container ran.

This way everyone on the team has the same container versions, no need to introduce any additional CI/CD steps for deploying new images to the registry and what not. It is also fast and productive. I can usually have my whole stack up and running locally in under a minute.

Post #3

egze

Thanks, I will explore volumes for deps. Although my idea with multi stages was supposed to take care about deps and libs. It would compile them in base, but then still run mix deps.get and compile in the next stage when you start it. So if mix.exs hasn’t changed - nothing would be done. But if you added anything new, it would add and compile only that.

But regardless, now I’m just curious why copying deps from base image didn’t work.

Post #4

akoutmos

Author of Build a Weather Station with Elixir and Nerves

Why even have a multistage build I guess is the question I have? Why not just base your container off of FROM registry.gitlab.com/username/project_name/base:latest? Seems like an extra step of indirection which doesn’t net you anything.

Post #5

egze

That‘s a good point. Honestly, I don‘t remember now, tried a couple of strategies today. I‘ll try it a bit later tonight and report back.

Post #6

cnck1387

Personally I use them in Phoenix (for development at least) for Webpack. I have a stage in my Dockerfile that does nothing but install Node and build assets. Then in a 2nd stage that sets up Elixir / Phoenix / mix, etc. I copy in the compiled assets into a folder.

Then I run a dedicated webpack service in Docker Compose but only in development (using an override file).

Haven’t deployed anything yet but I imagine once I tackle that, I’ll end up moving the webpack stage to nginx and also set up a 2nd stage for Elixir releases to base as my final prod image.

Post #7

akoutmos

Author of Build a Weather Station with Elixir and Nerves

That comment was aimed more at the OPs code sample where they used the registry image as a base, and then immediatly started the next stage:

FROM registry.gitlab.com/username/project_name/base:latest as base

FROM elixir:1.8-otp-22-alpine

For your case that sounds like a good solution as installing node+elixir in the same container can be annoying to do. When doing frontend work, I usually have 2 containers running, 1 for Elixir app and 1 for the JS app. Each container will then have read only access to the host filesystem (to the files that are relevant for each) and then kick off rebuilds as files change. At that point it is only a matter of setting up CORS correctly on the Phoenix app and my Vue SPA can communicate with my Phoenix API.

Post #8