Fl4m3Ph03n1x

Test function was called exactly X times

Background

I have some code that invokes a given function a certain number of times. I pass this function in the parameters so it is easy to stub.

However I need to know that under some conditions the function was called exactly X times. ExUnit.Case has an extremely limited support for this however, so I went ahead with the “send yourself a given message trick”

Code

    @tag :wip
    test "calls given function X times" do
      my_pid = self()

      deps = [
        lookup_fn: fn _key ->
          send(my_pid, :lookup)
          {:ok, 1}
        end
      ]

      MyApp.do_work(deps)
     # how to test I got the :lookup message exactly twice?
    end

Research

The first easy solution is using receive multiple times:

receive do
  msg -> assert msg == :lookup
end

receive do
  msg -> assert msg == :lookup
end

receive do
  msg -> assert msg == :lookup
end

# etc...

For example, if I wanted to check :lookup was called 10 times, I would have 10 receives.
This solution is very poor for 2 reasons:

A ton of code repetition
It doesn’t check that :lookup was called exactly X times, it checks it was received at least X times.

I searched for some libraries but couldn’t find anything. The closes thing I found was the SO question using macros:

Which I tried to adapt into an assert_receive_at_least (but failed miserably):

defmacro assert_receive_at_least(pattern, times, timeout \\ 500) do
    quote do

      defp loop(_pattern, current_times, total_times, _timeout) when current_times == total_times do
        {:ok, :received}
      end
      defp loop(pattern, current_times, total_times, timeout) do
        receive do
          msg -> assert pattern == msg
        after timeout -> {:error, :timeout}
        end
      end

      defp run(pattern, times, timeout) do
        loop(pattern, 0, times, timeout)
      end

      run(unquote(pattern), unquote(times), unquote(timeout))
    end
  end

Questions

How do I check if a given message was received X times?
How do I check if a given message was received at least X times?
Are there any libraries that add decent stub support ?

18 comments

#testing #exunit

0 6291 18

2019-02-27 16:59:29 UTC

Marked As Solved

LostKobrakai

x = 10
for _ <- 1..x, do: assert_received :lookup
refute_received :lookup

Post #1

Also Liked

peerreynders

Just an observation.

I personally never got into mocking. I only crossed paths with it about 5 years ago when using Mocha/Sinon/Chai. Given the nature of JavaScript (or Ruby for that matter) I can see how one could go down that road.

My exposure to microtesting goes back to the early days JUnit.

Mocking only became a thing some time later and when I looked into it in Java-land, a lot of it relied on reflection - my reaction:

Eeewh! That’s cheating.

Well, not entirely. It’s justified if you are mocking 3rd party dependencies but even then there was always the choice to just decouple that behind a façade.

That’s cheating.

That reaction was driven by the notion that mocking your own code removes the incentive to constantly assess and re-evaluate your boundaries. With classical testing there is the constant pressure to adjust your boundaries to be able to test what you want to test.

To me personally the primary benefit of Test Driven Development (in the classical sense) was that constant challenging of those boundaries (and the portals passing through them) - not the test first religion.

Of course even that can go off the rails as evidenced by the whole test induced design damage discussion.

One thing that isn’t clear from this discussion is:

why is it so important that the lookup is called 10 times that it needs to be tested at this level?
what is responsible for performing that lookup in the real system?

For example, if the lookup goes to another process anyway, wouldn’t it make more sense to test it with a fake process which keeps track of the invocation details that are then sent to the testing process?

Somewhat related in the JavaScript space: Please don’t mock me

Post #18

peerreynders

It’s already covered.

for _ <- 1…x, do: assert_received :lookup this will use assert_receive x times. If x receives do not happen, a timeout will fail the test.
refute_received :lookup this ensures that that there are no further receives beyond x for the specified timeout.

So combined the test ensures that exactly x messages were received within the specified timeouts.

Post #6

amatalai

Maybe GitHub - appunite/mockery: Simple mocking library for asynchronous testing in Elixir. · GitHub would be better for your needs. It works well if you don’t need to check function calls between different processes (disclaimer: I am the author of this library, I can be biased)

Post #5