How to cache erlang builds on CI?

Are you caching both the deps and build folders? On circle ci we’re caching them both separately

      # restore deps cache
      - restore_cache:
          keys:
            # CI_CACHE_VERSION is used to manually bust the cache
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}-{{ checksum "mix.lock" }}
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
            - messages-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-

      # restore build cache
      - restore_cache:
          keys:
            - messages-build-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
            - messages-build-v9-{{ .Environment.CI_CACHE_VERSION }}-

Then steps related to compiling and running migrations. After that we can save the cache:

      # save deps cache
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}-{{ checksum "mix.lock" }}
          paths: "deps"
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
          paths: "deps"
      - save_cache:
          key: my-app-deps-v9-{{ .Environment.CI_CACHE_VERSION }}-
          paths: "deps"

      # save build cache
      - save_cache:
          key: my-app-build-v9-{{ .Environment.CI_CACHE_VERSION }}-{{ .Branch }}
          paths: "_build"
      - save_cache:
          key: my-app-build-v9-{{ .Environment.CI_CACHE_VERSION }}-
          paths: "_build"

There’s multiple caches so that if for example the mix.lock changes then we can fallback to the next most recently used cache. Also the v9 is there so that when we make large changes we can cache bust everything and we can do something similar by changing the CI_CACHE_VERSION env variable (but we can change this without pushing a code update). A reason you’d need to bust the cache is when a library reads from application config during compilation (a great example is the Elixir mime type plug: https://hexdocs.pm/mime/MIME.html).

Here’s the official CircleCi elixir guide (although I don’t find it super helpful): https://circleci.com/docs/2.0/language-elixir/

I hope that helps!

1 Like

Thanks for the response axelson!

The config you posted is very close to EXACTLY what we are using :). As I stated above, this works for caching ELIXIR build files.

It does NOT work for caching the built ERLANG libs. I suspect this is because restoring a cache does not preserve the original file modification times - see mix compile docs and this circleci question.

If you watch your build logs, I think you will find that your tasks are rebuilding the erlang libs every time too.

I am trying to find a way to tell mix that it doesn’t need to recompile the erlang libs.

As I already said, you need to tell rebar not to recompile, as mix simply asks the dependencies manager if it needs to be recompiled.

Nobbz: Thanks for your reply! Any clues on how I can do that?

AFAIK you currently can’t, therefore I suggested to get in touch with the rebar team in How to cache erlang builds on CI?

Thanks NobbZ

I opened this issue on the rebar3 repo: https://github.com/erlang/rebar3/issues/1824

1 Like

@samphilipd were you eventually able to resolve this?

I ‘think’ if you just touch every compiled/beam file then it won’t recompile them due to how rebar works? I ‘think’? >.>

I tried:

touch _build/test/lib/*/ebin/*
touch _build/test/lib/*/.mix/*
touch _build/test/lib/*/include/*
touch _build/test/lib/*/consolidated/*
touch _build/test/lib/*/mix.rebar.config

Still recompiles everything, not sure if those are the files and if someone were able to solve. @axelson, did you found out?

1 Like

We don’t currently have any direct erlang dependencies so unfortunately I’m not that much help at the moment.

I see, but I guess the problem happens for indirect erlang deps too. Just mentioning…

Related: Are there people that have experience in doing the same thing (caching Elixir and/or Erlang dependencies so that only changed things are re-compiled) for the Travis CI?

Currently the Travis builds take almost four minutes, of which running the actual test suite takes only about 15 seconds.

This Travis CI runs each of the test suites in just over a minute. The linting and dialyzer tasks are ran as jobs after the tests pass, which really speeds things up.

1 Like

:wave:

Just ran into a somewhat similar issue (erlang deps didn’t seem to be cached) and in my case the problem was in not caching deps folder after running mix compile. This is important since rebar3 actually stores ebins in deps, and in _build mix only creates symlinks.

So, my approach before (which had the same problem as in OP):

# deps stage
- cache pull deps
- mix deps.get
- cache push deps

# compile stage
- cache pull deps, _build
- mix compile
- cache push _build

and my approach now:

# deps stage
- cache pull deps
- mix deps.get
- cache push deps

# compile stage
- cache pull deps, _build
- mix compile
- cache push deps, _build # <-- !!!
4 Likes

So are you running the “cache pull deps” step twice?

Once for each stage, yes. Otherwise mix complains about missing dependencies.

2 Likes

I can confirm that caching deps in addition to _build after running mix compile does result in no more recompilations. Lead to nice speed boost in our CircleCI builds.

Is this something which should be brought up with the rebar team? I think it’s counter intuitive that rebar places in deps on recompilation.

We were made aware of this recently and got it fixed. Didn’t realize all compilation wasn’t done to deps by mix :slight_smile:

The rebar3 version installed by mix should now be 3.13.1 which supports an additional argument of where to send output.

I’m not sure what version of mix uses the new output option… Will go look in a bit.

2 Likes

I’m not seeing a patch in elixir repo actually, so this may not be supported yet, but it is supported in rebar3.

Verified this is not yet supported in mix and likely won’t get attention until the end of May.

So if someone has time and wants to patch mix before then it’d be a great help. The change is to add -o <outputdir> where outputdir is the path to the dep under _build/ to the rebar3 bare compile command mix runs, if rebar3 version is >= 3.13.1 (or error if it fails with “invalid option” and tell the user to upgrade rebar3 I guess).

2 Likes