Mix test --cover is crashing with ** (EXIT from #PID...)

Questions / Help
1

Hi.

I’m trying to get better coverage for a project at work but mix test --cover keeps crashing

Erlang: 27.3
Elixir 1.18.3

$ mix test --cover

   ...
   100.00% | ProjectWeb.Dashboard
   100.00% | ProjectWeb.Endpoint
   100.00% | ProjectWeb.ErrorJSON
   100.00% | ProjectWeb.Gettext
-----------|--------------------------
    14.84% | Total

Coverage test failed, threshold not met:

    Coverage:   14.84%
    Threshold:  90.00%

** (EXIT from #PID<0.94.0>) {:file, ~c"cover/Elixir.Project.AuthOverrides.html", :emfile}

Everytime I rerun the test it will crash and output a new file. After a couple of tries it outputs this instead

    ...
    14.84% | Total

Coverage test failed, threshold not met:

    Coverage:   14.84%
    Threshold:  90.00%

Runtime terminating during boot ({undef,[{'Elixir.Inspect.Tuple',inspect,[{file,"/Users/evikstrom/src/tv4/playback3/test/support/conn_case.ex",emfile},#{safe=>true,base=>decimal,limit=>50,width=>80,'__struct__'=>'Elixir.Inspect.Opts',pretty=>false,syntax_colors=>[],charlists=>infer,binaries=>infer,char_lists=>infer,custom_options=>[],inspect_fun=>fun 'Elixir.Inspect':inspect/2,printable_limit=>4096,structs=>true}],[]},{'Elixir.Kernel',inspect,2,[{file,"lib/kernel.ex"},{line,2376}]},{'Elixir.Exception',format_banner,3,[{file,"lib/exception.ex"},{line,148}]},{'Elixir.Kernel.CLI',format_error,3,[{file,"lib/kernel/cli.ex"},{line,108}]},{'Elixir.Kernel.CLI',print_error,3,[{file,"lib/kernel/cli.ex"},{line,178}]},{'Elixir.Kernel.CLI',exec_fun,2,[{file,"lib/kernel/cli.ex"},{line,166}]},{'Elixir.Kernel.CLI',run,1,[{file,"lib/kernel/cli.ex"},{line,55}]},{init,start_it,1,[]},{init,start_em,1,[]},{init,do_boot,3,[]}]})

I have tries to switch Elixir and Erlang versions to no success.

Do anyone know what I’m doing wrong here?

2

Although I cannot claim to be sure, my understanding is that it crashes because the return value for when the test fails is 2:

Have you tried the command:

mix test --cover --exit-status 0

My assumption here being that if the return value is 0, the process will simply assume everything went well and wont therefore consider the exit a faulty one.

3

It seems like I get the same results.

4

To further test this I created a test project with the following commands and code:

❯ mix new test_cover
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/test_cover.ex
* creating test
* creating test/test_helper.exs
* creating test/test_cover_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd test_cover
    mix test

Run "mix help" for more commands.

~/fl4m3 
❯ cd test_cover/

~/fl4m3/test_cover is 📦 v0.1.0 via 💧 v1.18.2 (OTP 27) 
❯ mix test --cover
Compiling 1 file (.ex)
Generated test_cover app
Cover compiling modules ...
Running ExUnit with seed: 69712, max_cases: 32

..
Finished in 0.04 seconds (0.00s async, 0.04s sync)
1 doctest, 1 test, 0 failures

Generating cover results ...

Percentage | Module
-----------|--------------------------
    20.00% | TestCover
-----------|--------------------------
    20.00% | Total

Coverage test failed, threshold not met:

    Coverage:   20.00%
    Threshold:  90.00%

Generated HTML coverage results in "cover" directory

The code is as follows, in order to fail the coverage:

defmodule TestCover do
  @moduledoc """
  Documentation for `TestCover`.
  """

  @doc """
  Hello world.

  ## Examples

      iex> TestCover.hello()
      :world

  """
  def hello do
    :world
  end

  def bye(lang) do
    case lang do
      :es -> "adios"
      :en -> "bye"
      :ca -> "adeu"
    end
  end
end

The interesting part in all of this is the last line from the bash:

Generated HTML coverage results in "cover" directory

And indeed I have a cover directory with a file inside.

The logs you showed do now have that line. This leads me to believe that the file or the folder are not being created or that there is an issue when doing it.

With this in mind, I invite you to play around with:

--export-coverage - the name of the file to export coverage results to. Only has an effect when used with --cover

this flag, and try to change the output directory. You can use a full path to a place you know you have access to.

You can also check if:

  • do you have the cover folder?
  • is a file created inside that folder?
  • do you (the process running) has to correct write permissions?
5

Have you recently updated Erlang or Elixir?

Try:

$ rm -rf _build/ deps/
$ mix deps.get
$ mix deps.compile
$ mix test --cover

In that order.

2 Likes
6

Hi, sorry for late reply. Never managed to get it to work, started to use excoveralls instead.