How to accelerate docker build?

Questions / Help
,
1

Hi all,

I’m maintaining a faily large Elixir project (100 direct dependencies, 177MB deps, 554 source files, 56MB _build/dev).

I release it as a docker image.

My issue is that docker build is very long as it has to download dependencies, build dependencies, build sources, each time.

I’m looking for tips to accelerate this step, maybe using cache, since dependencies do not vary much and sources are largely the same from one build to the next.

I already reordered instruction to avoid docker cache invalidation if nothing changes (as explained here). But if only a tiny piece is change (eg: new version of a deps), the whole layer is invalidated.

Any idea ?

2

Just to narrow the list of suspects first: you’re using a multi-stage Dockerfile, right? And mix deps.get and mix deps.compile are both separate commands, not mixed with others on the same line, correct?

Oh, and this: are you cross-compiling for another CPU architecture?

1 Like
3

you’re using a multi-stage Dockerfile, right?

Yes

mix deps.get and mix deps.compile are both separate commands

Yes

are you cross-compiling for another CPU architecture?

Nope

4

Nothing quick jumps to mind then. I suggest you post your Dockerfile so others can chime in.

5

If you have the time to invest into learning Nix, you can build Docker containers with Nix.

EDIT: Updated to a higher quality video

6

Thanks ! Look promising. Plus I always wanted to have a try at grasping nix (it is on my machine but I don’t use it much yet). I’ll make some experiment and post result here.

PR : for those who want to spare time, the meat of the talk starts at 25:07.

7

A general approach in situations like this is to split the work that’s done in the layer into pieces that can be individually invalidated.

When I saw a similar issue with a Rails project (gems with native extensions take a long time to install), the solution was to have a separate step that only installed those gems before the main bundle install. That was simpler to implement because the gems were going to the system location (versus building in a Mix project).

Instead of a single mix deps.get and mix deps.compile, you could have a “mix-foundation.exs” (with lockfile set to mix-foundation.lock) then use MIX_EXS=mix-foundation.exs mix deps.get etc

That should produce a rarely-invalidated layer that pre-populates the _build directory and speeds up the plain mix deps.compile later.

One thing to watch out for: if the versions in the main file drift away from the ones in the foundation, nothing will fail but build times will go up when the correct versions have to also be installed. (the Ruby version had been in place for years and had this exact drift)

3 Likes