Doesn’t the compilation get cached?

It doesn’t, after running time mix test some_test_file.exs --cover multiple times,

Test output:

Finished in 0.1 seconds (0.1s async, 0.00s sync)
22 tests, 0 failures

Time output:

mix test some_test_file.exs --cover  58.22s user 12.81s system 449% cpu 15.812 total

:cover compiling takes 7 seconds, and then the rest of the time is spent on creating the summary and HTML generation. On CI --cover adds a whole minute, which is the main reason I wanted to enable this native coverage.

I did make some progress, but still didn’t manage to make it work

Environment variables

export ERL_COMPILER_OPTIONS="[line_coverage,force_line_counters]"
export ELIXIR_ERL_OPTIONS="+JPcover line_counters"

    dbg :code.coverage_support() # true
    dbg :code.get_coverage_mode() # :line_counters
    dbg :code.get_coverage_mode(mod) # :line_counters
    dbg mod.module_info()[:compile][:options] # [:no_spawn_compiler_process, :from_core, :no_core_prepare, :no_auto_import, :line_coverage, :force_line_counters]
    dbg :code.get_coverage(:line, mod)  # []

:code.get_coverage is returning an empty list, but if I init :cover it returns the expected data.

I believe cover will use native coverage by default. The native coverage speeds up the time for running tests, it may not necessarily speed up the time for compiling nor building the reports.

Do you see any executable_line instructions if you disassemble the compiled module?

This should be the way to do it from Elixir:

:beam_disasm.file(mod)

Here is an extract from a disassembly of one of my Erlang modules:

     [{function,t,1,2,
                      [{label,1},
                       {line,1},
                       {func_info,{atom,t},{atom,t},1},
                       {label,2},
                       {executable_line,2,1},
                       {executable_line,3,4},
                       {call_only,1,{t,'-t/1-lc$^0/1-0-',1}}]},

Yes, cover uses native coverage on systems that support it.

:beam_disasm.file(:code.which(mod)) doesn’t have executable_line instructions, is there any other configuration that I need to add to add them?

No, that’s correct. That will tell the Erlang compiler to instrument the code. However, it seems that the way the Elixir compiler invokes the Erlang compiler, line_coverage instrumentation will not work for Elixir code.

To confirm, I changed your setup to:

export ERL_COMPILER_OPTIONS=[line_coverage,force_line_counters,time]

That will instruct the Erlang compiler to print the name of each pass it executes.

If I then compile an Erlang module, the first few passes will be shown as follows:

Eshell V15.2.4 (press Ctrl+G to abort, type help(). for help)
1> c(t).
Compiling t.erl
 remove_file                   :      0.000 s       0.9 kB
 parse_module                  :      0.006 s       2.5 kB
 transform_module              :      0.000 s       2.5 kB
 lint_module                   :      0.005 s       2.5 kB
 beam_docs                     :      0.002 s       2.5 kB
 remove_doc_attributes         :      0.000 s       2.5 kB
 compile_directives            :      0.000 s       2.5 kB
 sys_coverage                  :      0.001 s       2.8 kB
 expand_records                :      0.002 s       2.8 kB
 core                          :      0.007 s      20.9 kB
 sys_core_fold                 :      0.003 s      14.7 kB
 sys_core_alias                :      0.001 s      14.7 kB

The sys_coverage pass is the pass that adds the executable_line instructions.

If I compile using the Elixir compiler, the sys_coverage pass is not run:

ex(1)> c "test.ex"
Compiling /Users/bjorng/git/otp/test.ex
 get_module_name_from_core     :      0.000 s      25.5 kB
 core_lint_module              :      0.000 s      25.8 kB
 sys_core_fold                 :      0.000 s      20.9 kB
 sys_core_alias                :      0.000 s      20.9 kB
 core_transforms               :      0.000 s      20.9 kB

That’s why there are no executable_line instructions in the generated BEAM code.