MecksUnit: Elegantly mock module functions in (async) ExUnit tests

exunit
asynchronous
mocking
meck

#1

It is a well-know topic within the Elixir community: “To mock or not to mock? :)”

Every alchemist probably has his / her own opinion concerning this topic. José Valim and Plataformatec has published the Hex package Mox which complies with his article on mocking in Elixir.

Personally, I’m not convinced in having to change the code “in service of” testing certain modules. Why would one add abstraction to code of which its purpose isn’t supposed to be interchangeable (with mock modules for instance)?

After some Googling, I found Espec of which I thought that that’s a little bit too much. Finally, I found Mock which could have done the job. But there are two downsides:

  1. You cannot use async: true
  2. Defining the mock functions could have been done in a more readable way

Based on that, I decided to write MecksUnit which solves just that. An example:

defmodule Foo do
  def trim(string) do
    String.trim(string)
  end
end

defmodule MecksUnitTest do
  use ExUnit.Case, async: true
  use MecksUnit.Case

  defmock String do
    def trim("  Paul  "), do: "Engel"
    def trim("  Foo  ", "!"), do: "Bar"
    def trim(_, "!"), do: {:passthrough, ["  Surprise!  !!!!", "!"]}
    def trim(_, _), do: :passthrough
  end

  defmock List do
    def wrap(:foo), do: [1, 2, 3, 4]
  end

  mocked_test "using mocked module functions" do
    task =
      Task.async(fn ->
        assert "Engel" == String.trim("  Paul  ")
        assert "Engel" == Foo.trim("  Paul  ")
        assert "Bar" == String.trim("  Foo  ", "!")
        assert "  Surprise!  " == String.trim("  Paul  ", "!")
        assert "MecksUnit" == String.trim("  MecksUnit  ")
        assert "Paul Engel" == String.trim("  Paul Engel  ", " ")
        assert [1, 2, 3, 4] == List.wrap(:foo)
        assert [] == List.wrap(nil)
        assert [:bar] == List.wrap(:bar)
        assert [:foo, :bar] == List.wrap([:foo, :bar])
      end)

    Task.await(task)
  end

  test "using the original module functions" do
    task =
      Task.async(fn ->
        assert "Paul" == String.trim("  Paul  ")
        assert "Paul" == Foo.trim("  Paul  ")
        assert "  Foo  " == String.trim("  Foo  ", "!")
        assert "  Paul  " == String.trim("  Paul  ", "!")
        assert "MecksUnit" == String.trim("  MecksUnit  ")
        assert "Paul Engel" == String.trim("  Paul Engel  ", " ")
        assert [:foo] == List.wrap(:foo)
        assert [] == List.wrap(nil)
        assert [:bar] == List.wrap(:bar)
        assert [:foo, :bar] == List.wrap([:foo, :bar])
      end)

    Task.await(task)
  end

  defmock String do
    def trim("  Paul  "), do: "PAUL :)"
  end

  defmock List do
    def wrap([1, 2, 3, 4]), do: [5, 6, 7, 8]
    def wrap(nil), do: ~w(Surprise)
  end

  mocked_test "using different mocked module functions" do
    task =
      Task.async(fn ->
        assert "PAUL :)" == String.trim("  Paul  ")
        assert "PAUL :)" == Foo.trim("  Paul  ")
        assert "  Foo  " == String.trim("  Foo  ", "!")
        assert "  Paul  " == String.trim("  Paul  ", "!")
        assert "MecksUnit" == String.trim("  MecksUnit  ")
        assert "Paul Engel" == String.trim("  Paul Engel  ", " ")
        assert [:foo] == List.wrap(:foo)
        assert ["Surprise"] == List.wrap(nil)
        assert [:bar] == List.wrap(:bar)
        assert [:foo, :bar] == List.wrap([:foo, :bar])
        assert [5, 6, 7, 8] == List.wrap([1, 2, 3, 4])
      end)

    Task.await(task)
  end
end

Mocking module functions is pretty straightforward and done as follows:

  1. Add use MecksUnit.Case at the beginning of your test file
  2. Use defmock as if you would define the original module with defmodule containing mocked functions
  3. Use mocked_test as if you would define a normal ExUnit test after having defined all the required mock modules

The defined mock modules only apply to the first mocked_test encountered. So they are isolated (despite of :meck having an unfortunate global effect ) as MecksUnit takes care of it. Also, non-matching function heads within the mock module will result in invoking the original module function as well. And last but not least: you can just run the tests asynchronously .

Enjoy using MecksUnit (if you prefer unobtrusive mocking). A Github star is very welcome, haha :wink:


#3

Released MecksUnit v0.1.2 in which you can assert function calls with either called (returns a boolean) or assert_called (raises an error when not having found a match):

assert called List.wrap(:foo)
assert_called String.trim(_)

#4

MecksUnit v0.1.3 is out. It includes the fix for :meck related compile errors which often occurred when mocking within multiple files


#5

Hi, did you think, by chance, to add a mocked_test macro that would be compatible with Phoenix test cases? So that you can write then

mocked_test "test name", %{conn: conn} do
 test_body()
 and_assertions()
end

Because If I remember correctly, mocked_test does not accept the second argument with a map returned by setup block.


#6

Sure, one moment :sweat_smile:


#7

Just released MecksUnit v0.1.4 :slight_smile:


#8

I don’t know what happened but since 1.3 mocked_test fail with same defmock do block. Instead of mocking it’s passing through function.
Nevermind, I forgot to do MecksUnit.mock() Working like a charm :+1:


#9

There is some conflict with https://github.com/parroty/excoveralls, because when you run tests with coveralls and add some options, the mock is not registered, and test uses original module instead of mocked one
example command that fails mix coveralls.html -u --exclude not_implemented But I was unable to pinpoint the cause.

It works OK with MecksUnit 1.2 but not with 1.3 there must be some change there that makes it fail.


#10

Lol. I have been debugging excoveralls related issues myself at the moment. I’m almost there, just need to figure out on how to hook in after the test suite has finished and just before excoveralls does his thing. Stay tuned, I hope to have it solved within a few hours.


#11

Well, after some head banging on my desk table whilst digging into the dark caves of ExCoveralls, ExUnit, :meck and :cover I have managed to fix the ExCoveralls related errors :sunglasses:

In other words, MecksUnit v0.1.5 should solve your problems! Please let me know whether that is actually the case or not :muscle:


#12

Somehow I’m still unable to run it with phoenix controller tests. I’ve spent on it only few minutes though, because of time constrains this week. with MecksUnit.mock() in test_helper.exs and

  use MerchantWeb.ConnCase, async: true
  use MecksUnit.Case

  defmock LedgerService do
    def init(_authorization, _uuid), do: :ok
  end

In my test file i get error from the original LedgerService module and then

20:59:51.475 [error] GenServer #PID<0.667.0> terminating
** (stop) {:not_mocked, LedgerService}
    (meck) /home/sztosz/Documents/app-umbrella/deps/meck/src/meck_proc.erl:467: :meck_proc.gen_server/3
    (meck) /home/sztosz/Documents/app-umbrella/deps/meck/src/meck.erl:475: :meck.unload/1
    (elixir) lib/enum.ex:765: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:765: Enum.each/2
    (mecks_unit) lib/mecks_unit/unloader.ex:21: MecksUnit.Unloader.handle_cast/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", {:suite_finished, 8043604, nil}}
20:59:51.479 [error] Task #PID<0.630.0> started from #PID<0.92.0> terminating
** (stop) exited in: GenServer.stop(#PID<0.667.0>, :normal, 30000)
    ** (EXIT) exited in: :sys.terminate(#PID<0.667.0>, :normal, :infinity)
        ** (EXIT) an exception was raised:
            ** (ErlangError) Erlang error: {:not_mocked, LedgerService}
                (meck) /home/sztosz/Documents/app-umbrella/deps/meck/src/meck_proc.erl:467: :meck_proc.gen_server/3
                (meck) /home/sztosz/Documents/app-umbrella/deps/meck/src/meck.erl:475: :meck.unload/1
                (elixir) lib/enum.ex:765: Enum."-each/2-lists^foreach/1-0-"/2
                (elixir) lib/enum.ex:765: Enum.each/2
                (mecks_unit) lib/mecks_unit/unloader.ex:21: MecksUnit.Unloader.handle_cast/2
                (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
                (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
                (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
    (elixir) lib/gen_server.ex:883: GenServer.stop/3
    (ex_unit) lib/ex_unit/event_manager.ex:21: anonymous fn/2 in ExUnit.EventManager.stop/1
    (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ex_unit) lib/ex_unit/event_manager.ex:20: ExUnit.EventManager.stop/1
    (ex_unit) lib/ex_unit/runner.ex:23: ExUnit.Runner.run/2
    (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
    (elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: &ExUnit.run/0
    Args: []

But when I run single test, or just test in one file the tests pass.

When that one whole app from that umbrella or all apps in umbrella it fails.


#13

It is not the umbrella app which causes the problem (successfully using MecksUnit with an umbrella app myself).

A new version has been released. Tackling the error when trying to unload a non-mocked module. Your problem should be solved with MecksUnit v0.1.6. Please confirm, ok?


#14

Now I’m not sure what’s going on. The error goes away, but I either get the error or not depending on test seed.

mix test --seed 57991 test/merchant_web/
...
Finished in 8.1 seconds
2 doctests, 92 tests, 0 failures
mix test --seed 398178 test/merchant_web/

  1) test POST /merchant/register creates Company, User (owner), Permission and CompanyCurrency (MerchantWeb.UserControllerTest)
     test/merchant_web/controllers/user_controller_test.exs:134
     ** (CaseClauseError) no case clause matching: []
     code: |> post("/merchant/register", params)
     stacktrace:
       (hackney) /home/sztosz/Documents/Bitt/mmoney-umbrella/deps/hackney/src/hackney_url.erl:211: :hackney_url.parse_netloc/2
       (hackney) /home/sztosz/Documents/Bitt/mmoney-umbrella/deps/hackney/src/hackney.erl:345: :hackney.request/5
       (httpoison) lib/httpoison/base.ex:630: HTTPoison.Base.request/9
       (ledger_service) lib/ledger_service.ex:131: LedgerService.init/2



Finished in 8.1 seconds
2 doctests, 92 tests, 1 failure

I omitted the stack trace, because it’s irrelevant, but honestly I have no clue why with some seeds the mocked module is used in test, and why with other seeds test uses original module.


#16

OK, i don’t know what was the real issue, I changed code so many times, I changed between versions of MecksUnit, I changed between MecksUnit and Mock many times. After last change of failing test from Mock to MecksUnit it started to work. Just like that.

So what I wanted to write is, thank you for your changes, after all it works Like I need it to work, and what’s most important… It works :slight_smile:

UPDATE: Actually random seeds still make tests fail (those using the mocks) and other don’t. ¯_(ツ)_/¯


#17

You are welcome (for what it’s worth).

Can you please try to reproduce it in such a way that I have a case which I need to solve? I can imagine that you sharing your code is not what you want. So a smaller (non-valuable) case would be better.

Really hoping that you can deliver such a small case


#18

I moved to Mox after being unable to find the issue. If I find time next week I’ll try to create something to that will reproduce this strange behaviour. But really don’t know what can be the cause. And I can’t share the original code.


#19

That’s sad to hear :’(


#20

Hi! This looks really interesting.

By that you mean that I can/need to define multiple times the mocked module based on the mocked_test?
Can I define mocks inside describe blocks?

Thanks for your contribution!!!


#21

I’m not sure, but this may have been what caused my tests to fail. :thinking: Because I had many test files with many tests in them each, and few of those files were mocking same function from the same module.


#22

The amount of files mocking the same function should not be a problem (MecksUnit makes it one mocked function). Maybe you could provide a pseudo test file with a similar structure in terms of tests, setup, etc.