Order of copying files in Dockerfile (cache invalidation of built deps)

Questions / Help
, ,
1

Hi,

I have the following order of copying/invalidation in my Dockerfile:

COPY config/config.exs config/config.exs
COPY config/prod.exs config/prod.exs
RUN mix deps.compile

This was based upon an unfounded belief that the config.exs/prod.exs could influence compile time options in deps - as built by mix deps.compile.

It’s been pointed out to me that my assumption is incorrect, they only affect the building of artefacts in lib and test.

Is this true? Is it worth invalidating the built dependencies every time I change the config files?

2

Of course those changes can affect the compilation result of your dependencies, though compiletime configuration like that is frowned upon for libraries.

2 Likes
3

Ok - I’ll keep it so and err on the side of caution. Thanks man.

4

Yeah, config files are necessary before you call mix deps.compile or you’ll get weird errors.

I recently optimised the Dockerfile for my umbrella app and this is what I found to be the most optimal that also compiles ONLY what has changed:

# Copy only mix files, so dependencies are cached and recompiled only iff 
# dependencies or config change
COPY mix.lock mix.exs /build/
COPY config /build/config/
COPY apps/inner_app1/mix.* /build/apps/inner_app1/
COPY apps/inner_app2/mix.* /build/apps/inner_app2/

WORKDIR /build
RUN mix do deps.get --only prod, deps.compile

# Build assets - required to be done at this step as some Phoenix JS deps 
# actually reference mix packages
COPY apps/inner_app1/assets /build/apps/inner_app1/assets/
WORKDIR /build/apps/inner_app1/assets
RUN yarn install --frozen-lockfile && yarn cache clean && yarn run build:prod

# Compile the app proper. This step is the one that
# gets invalidated the most often.
WORKDIR /build
COPY . /build
RUN mix compile

Note: I only have a top-level config directory. If your umbrella apps have their own config, you should still copy them over before doing mix deps.compile

5

Thanks @1player - yep I’ve never been a fan of umbrella apps and when it comes to containerisation things get even more complicated due to the multiple locations where one can unnecessarily invalidate the build.

@1player - looking at the configuration you’ve shared (many thanks) I would also recommend :

  • setting the ENV variable MIX_ENV=prod and only copying the config/config.exs and config/prod.exs into the image - this will save your image from being invalidated by tweaks to the test.exs and develop.exs file
  • set WORKDIR at the very start of the image build, to something like /app/build - makes it less verbose and reduces cognitive overhead for later maintainers.
1 Like