Gitlab CI rebuilds umbrella, despite build in previous job

Questions / Help
, ,
1

We are curretnly trying to reduce the amount of compute units used by our CI.

Thats why I split out mix deps.get && mix compile into a job that produces artifacts, which then can get used in the follow up jobs to save up some compute time.

This seems to work well in theory, and as we have 5 jobs based on the build output, we expect roughly 3 equivalents of mix compile to be saved.

Sadly, it appears that in the subsequent jobs, the project itself is still built each time.

So far, the effect of extracting the build into a previous step is therefore barely better than caching.

To comparision see this, where the 2 apps in the umbrella are built anew:

While the previous step does properly build it here as well:

The downloaded artifacts have all the BEAM modules and application manfests as I would expect them.

Has anyone an idea how to get this working, without doing the repeated compilation of the projects application again and again?

This repeated builds of the project members seems to be one of the most costly factors in the CI.

1 Like
2

I’ve noticed this too happening in my CI pipelines. I have the feeling that this is related to the rebar3. In my logs I could see recompilation of libraries that were not part of mix.

In a few OSS projects, I saw that some folk was caching .mix (in your case /root/.mix/) folder too, have you tried doing that?

1 Like
3

Thanks for the input, sadly this doesn’t change much, the recompilation in the later jobs still happens, while at the same time, the build step succesfully builds nothing if no actual code changes happened (but the steps after it still do!)