Problem Using ExCoveralls When There Are Specs In Code

Questions / Help
1

In short, when the specs are removed from lib/excoveralls_spec_lab/kata/user.ex , then the command mix coveralls.html works without a problem and generates the output html file cover/excoveralls.html .

But if there are any specs in lib/excoveralls_spec_lab/kata/user.ex , then the command mix coveralls.html will show this error:

** (exit) an exception was raised:
    ** (MatchError) no match of right hand side value: :error
        cover.erl:1591: :cover.do_compile_beam2/6
        cover.erl:1483: :cover.do_compile_beam/3
        (stdlib) lists.erl:1239: :lists.map/2
        cover.erl:2911: anonymous fn/2 in :cover.pmap_spawn/4
    cover.erl:600: :cover.call/1
    lib/excoveralls.ex:32: ExCoveralls.start/2
    (mix) lib/mix/tasks/test.ex:351: Mix.Tasks.Test.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    lib/mix/tasks.ex:54: Mix.Tasks.Coveralls.do_run/2
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2

Elixir and Erlang versions:

erlang 22.0.1
elixir 1.8.2

Sample repository for reproducing the issue can be found here.

An issue is also opened for this.

2

As already noted in the issue you’re defining specs with a non existing type.

3

Which type(s) does not exist? (I could not find the non-existing types in the issue)

Added a comment in the issue. Added dialyxir and it provides a better error that states there are Unknown Types.

4

There is no type ExcoverallsSpecLab.Kata.User.t() like @Eiji said on the github issue. You need to define it. While it is common to use t as type name for the struct a module defines it is just a convention and not defined automatically.

4 Likes
5

Hi, thanks for this discussion, as it seems I am not the only one with this issue. I’m getting the same type of error in an umbrella project. I have 4 apps in there, things are working fine, but one of the apps crashes.
here’s my circleci job:

version: 2.1
jobs:
  test:
    docker:
      # Codecov uses bash, and bash is not installed on alpine containers.
      - image: elixir:1.9
      - image: circleci/postgres:11
      - image: rabbitmq:3.7-alpine
    environment:
      MIX_ENV: test
    steps:
      - checkout
      - run: mix do local.hex --force, local.rebar --force
      - run: mix do deps.get --only test, deps.compile, compile
      - run: mix do ecto.create --quiet, ecto.migrate
      - run: mix coveralls.json
      - run: bash <(curl -s https://codecov.io/bash)

Coveralls is properly configured (it worked fine before). The only thing that is different is that I’m using the CQRS/ES framework commanded from @slashdotdash . It fails both on CI and locally, pointing again at the issue with the :cover module (however I’m running with erlang 22, not 17 as it’s stated in the github issue linked earlier in the thread).

The error I get is roughly the same as shown before:

==> domain
Cover compiling modules ...
** (exit) an exception was raised:
    ** (MatchError) no match of right hand side value: :error
        cover.erl:1591: :cover.do_compile_beam2/6
        cover.erl:1483: :cover.do_compile_beam/3
        (stdlib) lists.erl:1239: :lists.map/2
        cover.erl:2911: anonymous fn/2 in :cover.pmap_spawn/4
    cover.erl:600: :cover.call/1
    (mix) lib/mix/tasks/test.ex:12: Mix.Tasks.Test.Cover.start/2
    (mix) lib/mix/tasks/test.ex:407: Mix.Tasks.Test.do_run/3
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/project.ex:352: Mix.Project.in_project/4
    (elixir) lib/file.ex:1542: File.cd!/2
    (mix) lib/mix/task.ex:431: anonymous fn/4 in Mix.Task.recur/1
    (elixir) lib/enum.ex:1948: Enum."-reduce/3-lists^foldl/2-0-"/3
    (mix) lib/mix/task.ex:430: Mix.Task.recur/1
    (mix) lib/mix/project_stack.ex:208: Mix.ProjectStack.recur/1
    lib/mix/tasks.ex:54: Mix.Tasks.Coveralls.do_run/2
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
    (elixir) lib/code.ex:813: Code.require_file/2

In order to solve this, I had to remove all typespecs from my commanded app. This brings another question to me:

Where is the problem really coming from?

I can think of a few possiblities:

  • there still is an issue with the :cover erlang module
  • there is an issue with the Mix interoperability with the :cover erlang module
  • my typespecs were bad, although dialyzer was not complaining
  • there is an issue with the excoveralls package

I would love to move this problem forward, but I feel a bit lost now. Does anyone have an idea?

EDIT

I was able to fix the issue by declaring types in the files that were supposed to:

@type t ::%__MODULE__{}

putting this in the modules that required it fixed everything. I think it’s important to note that this was an issue as long as the missing types were used as arguments of the function being specified, things were fine if the return type of a function had not been specified at all in its module.

this is still a mystery to me, but at least I found a way to fix it.

5 Likes
6

I am having similar issues. However, I am getting the warning:

** (MatchError) no match of right hand side value: {:error, {:no_source_code_found, <module_name>}}

In the first module that this error was reported for, I commented out all @spec attributes, and that made the error go away for that module. The next module that generated the error has no @spec attributes. There is also no instance of <module_name>.t() referenced anywhere in the code. The module exists. It is aliased in a few places but no alias as instances. I’m not sure what to do to be able to run the test coverage tooling.

I am not using ExCoveralls and am just running mix test --cover. So I think this is an issue with whatever the interaction between Elixir, Mix, and :cover is.

If there is an actual issue, the error certainly could be improved. For the project, mix compile works fine with only a single (expected) warning, mix test passes, and mix credo shows no issues. I should note that the project is an umbrella project.

7

The function that returns that error is here:

My guess: something is producing compiled BEAM files in the wrong place or with the wrong compiler metadata (used by find_source), but only sometimes. “Deleting all the @specs” makes the problem vanish because it forces that file to be recompiled.

Edit to add: are you trying to calculate coverage across multiple apps in an umbrella project? The last time I tried tricking :cover into doing that, I got the whole BEAM to segfault…

8

We ran into a similar situation and the root cause was CI cache related. I just wrote a blog post about it.

9

I encountered the same problem in an umbrella project today, with no obvious change causing it.

I’ve found asking to export coverage fixed / worked-around the problem:

mix test --cover --export-coverage default

as indirectly suggested at mix test.coverage — Mix v1.11.2