Docker build upgrading packages

Questions / Help
, , ,
1

Hi all,

Has anyone experienced a Docker build decided to upgrade some packages by itself? My build was failing remotely (and locally when I tested) due to plug and plug_crypto being upgraded by the mix deps.get command, with the following output:

CleanShot 2022-12-12 at 16.43.56

I’m really confused as to what happened because my understanding is that mix deps.get will retrieve the locked versions? The mix.exs file was modified to use a :path for an unrelated package, but mix.lock was unchanged. The packages were within the permitted version constraints at all times.

The Dockerfile does the standard “dance” for efficiency reasons:

...
# Copy dependency config
COPY mix.* /app/

# Copy configs
COPY config /app/config

# install dependencies
RUN mix do deps.get, deps.compile

# Copy app
COPY . .

# build release
RUN \
  mkdir -p /app/build && \
  mix release --overwrite --path /app/build
...

I think the failure occurred because the final COPY . . reset mix.lock, causing a mismatch:

#25 1.000 * plug_crypto (Hex package)
#25 1.000   lock mismatch: the dependency is out of date. To fetch locked version run "mix deps.get"
#25 1.000 * plug (Hex package)
#25 1.000   lock mismatch: the dependency is out of date. To fetch locked version run "mix deps.get"
#25 1.005 ** (Mix) Can't continue due to errors on dependencies
#25 ERROR: process "/bin/sh -c mkdir -p /app/build &&   mix release --overwrite --path /app/build" did not complete successfully: exit code: 1

I ended up merging in the updated packages in order to fix the problem and now I can’t even reproduce it by going back to the old commits.

So I guess my question is, under what circumstances can mix deps.get choose to “upgrade” packages? For some reason I can’t even find the string where the “Upgraded:” prompt is output in mix.

This is on the hexpm/elixir:1.14.0-erlang-25.0.4-alpine-3.16.1 image so everything is pretty up-to-date.

Thanks for any insights!

2

What is the point of partial copy before doing mix deps.get? Ensure that all the project files are copied before you run any mix commands.

3

The partial copy is standard Docker practice because each instruction is cached. Any changes in the copied files blow the cache from that point on. Because dependencies change much more rarely than project code we want to retain this cache when possible. Without those lines, we’d be re-fetching all packages from Hex every time the project is built.

4

I would recommend you to cache in a different way, for example in Gitlab Runners that I use as the basis for my projects, I cache the deps folder after fetching them on my CI environment. On the other hand doing the way you are doing things can result in many bugs as mix tasks can also be compiled from the project and source files themselves.

5

Oh yeah mix deps.get does not have option to “fail if changes”. That feature will be introduced soon by Mix deps.get `--check-locked` parameter by Efesto · Pull Request #12184 · elixir-lang/elixir · GitHub (I had a step to check whether there is a dirty git change to prevent this)

You need to find out why it changes the lock file. Are you using the same elixir version between local dev (where you update mix.lock) and the docker build env? If you copy those files only and run it from local dev - then do you have the same problem? I’ve made some mistakes when copying files from local to docker env for instance.

1 Like