How to cache erlang builds on CI?

What do you mean by “compiles all the erlang modules”?

Are you compiling erlang or elixir from scratch in your build pipeline?

Also strategies for caching between subsequent builds or stages do depend on the system you use.

Jenkins with or without docker, gitlab-ci, travis or something completely different?

Are you targetting a docker container, LXS or even something completely different?

All those details affect the strategies you have available, which one you would use or how you use them.

We are using CircleCI (which runs all builds inside docker containers). The caching options available can save/restore any files, but I suspect it resets the timestamps on restore which might be causing this issue.

When I say “compiles all the Erlang modules”, I mean the output looks like this:

mix dialyzer --halt-exit-status
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Package unicode_util_compat-0.3.1 not found. Fetching registry updates and trying again...
===> Updating package registry...
===> Writing registry to /root/.cache/rebar3/hex/default/registry
===> Generating package index...
===> [appsignal:1.6.2], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.0], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.7.0-alpha.4], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.0-beta.1], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.3], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.7.0-alpha.3], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.7.0-alpha.2], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.7.0-alpha.1], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.5], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.1], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.4], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> [appsignal:1.6.0-alpha.1], Bad dependency version for httpoison: ~> 0.11 or ~> 1.0.
===> Writing index to /root/.cache/rebar3/hex/default/packages.idx
===> Compiling idna
===> Compiling ranch
==> poolboy (compile)
Compiled src/poolboy_worker.erl
Compiled src/poolboy_sup.erl
Compiled src/poolboy.erl
===> Compiling hpack
==> ssl_verify_fun (compile)
Compiled src/ssl_verify_util.erl
Compiled src/ssl_verify_fingerprint.erl
Compiled src/ssl_verify_pk.erl
Compiled src/ssl_verify_hostname.erl
===> Compiling certifi
===> Compiling hackney
===> Compiling cowlib
src/cow_multipart.erl:392: Warning: call to crypto:rand_bytes/1 will fail, since it was removed in 20.0; use crypto:strong_rand_bytes/1

===> Compiling cowboy
===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"4.1.0">>})
===> Downloaded package, caching at /root/.cache/rebar3/hex/default/packages/rebar3_hex-4.1.0.tar
===> Compiling rebar3_hex
===> Compiling redbug
...dialyzer output

Maybe you can have a container which only runs mix deps.compile? Which would only get rebuilt if mix.lock changes.

This is exactly what we do

The container runs mix deps.compile then copies _build/dev into a cache which is restored into subsequent containers.

The problem is for some reason, the Erlang builds are always marked as stale, so it always gets rebuilt regardless.

Not Erlang, id guess that a mix managed Erlang app wouldn’t recompile.

It’s rebar managed applications.

I think if you can manage to set up a pure Erlang project relying on rebar only which recompiles cached stuff on subsequent stages of the pipe, I’m sure the rebar people will be eager to help.

Are you caching both the deps and build folders? On circle ci we’re caching them both separately

      # restore deps cache
      - restore_cache:
          keys:
            # CI_CACHE_VERSION is used to manually bust the cache
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}-{{ checksum "mix.lock" }}
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-

      # restore build cache
      - restore_cache:
          keys:
            - messages-build-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
            - messages-build-v9-{{ .Environment.CI_CACHE_VERSION }}-

Then steps related to compiling and running migrations. After that we can save the cache:

      # save deps cache
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}-{{ checksum "mix.lock" }}
          paths: "deps"
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
          paths: "deps"
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-
          paths: "deps"

      # save build cache
      - save_cache:
          key: my-app-build-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
          paths: "_build"
      - save_cache:
          key: my-app-build-v9-{{ .Environment.CI_CACHE_VERSION }}-
          paths: "_build"

There’s multiple caches so that if for example the mix.lock changes then we can fallback to the next most recently used cache. Also the v9 is there so that when we make large changes we can cache bust everything and we can do something similar by changing the CI_CACHE_VERSION env variable (but we can change this without pushing a code update). A reason you’d need to bust the cache is when a library reads from application config during compilation (a great example is the Elixir mime type plug: MIME — mime v2.0.7).

Here’s the official CircleCi elixir guide (although I don’t find it super helpful): https://circleci.com/docs/2.0/language-elixir/

I hope that helps!

Thanks for the response axelson!

The config you posted is very close to EXACTLY what we are using :). As I stated above, this works for caching ELIXIR build files.

It does NOT work for caching the built ERLANG libs. I suspect this is because restoring a cache does not preserve the original file modification times - see mix compile docs and this circleci question.

If you watch your build logs, I think you will find that your tasks are rebuilding the erlang libs every time too.

I am trying to find a way to tell mix that it doesn’t need to recompile the erlang libs.

As I already said, you need to tell rebar not to recompile, as mix simply asks the dependencies manager if it needs to be recompiled.

Nobbz: Thanks for your reply! Any clues on how I can do that?

AFAIK you currently can’t, therefore I suggested to get in touch with the rebar team in How to cache erlang builds on CI? - #6 by NobbZ