Why does running `mix ecto.create` trigger a new compilation?

Phoenix Forum Questions / Help
, ,
1

In the process of setting up a Gitlab CI pipeline, I noticed something that I don’t understand.

My .gitlab-ci.yml looks like this:

variables:
  POSTGRES_HOST: "postgres"
  POSTGRES_DB: "postgres"
  POSTGRES_USER: "postgres"
  POSTGRES_PASSWORD: "postgres"
  MIX_ENV: "test"
  
stages:
  - build
  - test

default:
  image: elixir:1.9
  before_script:
    - mix local.hex --force
    - mix local.rebar --force

compile:
  stage: build
  script:
    - apt-get update
    - apt-get install -y postgresql-client
    - mix deps.get --only test
    - mix compile --warnings-as-errors
  artifacts:
    paths:
      - _build
      - deps
    expire_in: 1 week

lint:
  stage: test
  script:
    - mix format --check-formatted

test:
  stage: test
  services:
    - postgres:10.10
  script:
    - mix event_store.setup
    - mix ecto.create
    - mix ecto.migrate
    - mix test

Since jobs are run independently from each other, I save the compilation artifacts from the compile job to avoid recompiling in the next jobs. However, when the test job runs the mix ecto.create command, a compilation is triggered again. Why? Did not the compile job compile everything needed when running mix compile --warnings-as-errors?

2

Each mix env does have it’s own artefacts and mix test defaults to running in the test mix env, while other mix tasks, which don’t have a preferred env, default to dev.

3

Thank you for your quick reply LostKobrakai.

All commands are run for the test environment thanks to the variable in .gitlab-ci.yml

variables:
  MIX_ENV: "test"

Also, browsing the artifacts with Gitlab shows that there is only a test directory within the _build directory.

So I guess all mix tasks were run in the test env, right?

4

If the jobs are run independently are the clocks synchronized? Maybe the timestamps are out of sync which might trigger the recompilation

5

You do not “depend” on the previous stage, so there are no artifacts available during test stage.

6

Thank you for your answer. The Gitlab documentation about dependencies states:

Note that artifacts from all previous stages are passed by default.

So the artifacts should be passed. Also, the test job trace shows that the artifacts were downloaded:

Running with gitlab-runner 12.1.0 (de7731dd)
  on docker-auto-scale ed2dce3a
Using Docker executor with image elixir:1.9 ...
Starting service postgres:10.10 ...
Pulling docker image postgres:10.10 ...
Using docker image sha256:cf7301675e300ca20b74fb7fc3e34d2bfb04c6bc79699594217b0060e2312bcd for postgres:10.10 ...
Waiting for services to be up and running...
Pulling docker image elixir:1.9 ...
Using docker image sha256:f14e6364d0696d81769b31d6fb38f24b3bf10bf2503b005e5753cb6b10d746a4 for elixir:1.9 ...
Running on runner-ed2dce3a-project-13900485-concurrent-0 via runner-ed2dce3a-srm-1566462193-44c7355f...
Fetching changes...
Initialized empty Git repository in /builds/me/myapp/.git/
Created fresh repository.
From https://gitlab.com/me/myapp
 * [new branch]      master            -> origin/master
 * [new branch]      setup-ci-pipeline -> origin/setup-ci-pipeline
Checking out a7f07670 as setup-ci-pipeline...

Skipping Git submodules setup
Downloading artifacts for compile (277662661)...
Downloading artifacts from coordinator... ok        id=277662661 responseStatus=200 OK token=MDDAoYgU
$ mix local.hex --force
* creating /root/.mix/archives/hex-0.20.1
$ mix local.rebar --force
* creating /root/.mix/rebar
* creating /root/.mix/rebar3
$ mix event_store.setup
The EventStore database has been created.
The EventStore database has been initialized.
$ mix ecto.setup
Compiling 356 files (.ex)
Compiling lib/myapp_web/resolvers/accounts/change_user_role.ex (it's taking more than 15s)
[…]
Generated myapp app
The database for myapp.Repo has been created
The database for myapp.ReadStore has been created
$ mix test
Excluding tags: [:pending]

......................................................................... […]

Finished in 207.5 seconds
9 doctests, 3454 tests, 0 failures

Randomized with seed 555244
Job succeeded
7

I’ve never checked for that. I always explicitely define dependencies, to make sure to have the artifacts I want.

Anyway, mix does solely rely on the files timestamps when deciding wether or not a file/dependency needs recompilation.

The artifacts might be extracted and carry the original date, while git on a checkout always sets current time as cdate.

This is an assumption though. I usually do not care for rebuilding such stuff on subsequent stages (in elixir), as I have to recompile anyway for different environments…

You might try to recursively touch all artifacts, alternatively some mix tasks allow --no-compile or something like that, you might be able to play with that.

1 Like
8

Ok, that explains why a new compilation happens. Thank you