Test function was called exactly X times

x = 10
for _ <- 1..x, do: assert_received :lookup
refute_received :lookup

I would encourage you to think about the problem differently if possible - are there existing, concrete side effects you can observe directly after the function has been called N times? Perhaps several new DB rows get created, a counter gets incremented, etc.

If you’ve read Mocks and Explicit Contracts and still want to continue with this path knowing those trade-offs and accepting them, Mox is a great choice. May require some refactoring to establish your contracts as a behaviour.

Could you elaborate a little bit more on your solution?
Also, I assume that is the solution to question 1. Would you be kind enough to suggest something about the second question?

This is precisely the things I don’t want to test. I want to test that the messages of my system were sent to its collaborators. I don’t wanna know what they did. This is a hallmark principle of good testing and good design I won’t abandon so easily.

I have read that article. In fact that article is the reason why I am injecting the dependencies via functional injection. First, because not every code benefits from the added noise of a contract, and second because José Valim himself suggests in the article that functional dependency injection is better.

But even if I were to use Mox, it would still fall (very) short of the basic features I need to test, such as meaningful stub and spies.

There is a fairly subtle important distinction between “was this function called N times”, which is a blatant implementation detail, and “was this message delivered N times”, which is still an implementation detail, but one that is likely to survive longer within your business domain than a function name. You can test the latter without any external libraries, depending on how your code is structured. Presumably the pid or registered name of your collaborator appears somewhere within your function arguments, and you can point that to another listener in various ways that would allow you to assert on the messages received.

Can you mention some of the shortcomings you found when trying the library? Have you opened any GitHub issues on the project to have them considered? It supports defining stubs and mocks based on anonymous or captured functions, and it would be fairly trivial to pass from that back into a “real” behavioral module to observe that the functions were called but still test the real behavior too.

Maybe GitHub - appunite/mockery: Simple mocking library for asynchronous testing in Elixir. · GitHub would be better for your needs. It works well if you don’t need to check function calls between different processes (disclaimer: I am the author of this library, I can be biased)

It’s already covered.

• for _ <- 1…x, do: assert_received :lookup this will use assert_receive x times. If x receives do not happen, a timeout will fail the test.
• refute_received :lookup this ensures that that there are no further receives beyond x for the specified timeout.

So combined the test ensures that exactly x messages were received within the specified timeouts.

If you are using OTP 21+ then you can write it as:

    @tag :wip
    test "calls given function X times" do
      counter = :counters.new(1, [])

      deps = [
        lookup_fn: fn _key ->
          :counters.add(counter, 1, 1)

          {:ok, 1}
        end
      ]

      MyApp.do_work(deps)
      assert 2 == :counters.get(counter, 1)
    end

The function name is a detail, you are correct. I don’t care which specific function my code runs (if its name is A or B or whatever) but I do care that it runs the functions I inject on it a given number of times. This is the basics of testing (usually you test if a function was called once) and coulnd’t be farther from an implementation detail.

Allow me to showcase what a real test library is

https://sinonjs.org/releases/v7.2.4/

spies, stubs, fakes, mocks, timers, etc.
Admittedly they go out of the way with their own assertions (something which ExUnit could benefit from) but that’s another discussion.

A fair point, but one that I feel would be a waste of my time. I find it that Jose Valim would rather do other things then listening for a cry baby asking for features. Not only that, I need a solution I can use now, not a solution I can use in 4 months when the Elixir team finally has time to update Mox.

Ultimately, even though you are 100% logically correct, I still don’t engage in that course of action because I have a strong believe my efforts would be fruitless.

From the documentation I see a couple of profound limitations:

1. It still forces you to use contracts and works only via mocks. As I mentioned before, not everything benefits from a redundant interface.
2. It really doesn’t allow the stub to return a different result on each call.

If I am incorrect, please feel free to en-light me

Checking it, thanks for popping in !

Using a global state ETS table on my tests truly is something I frown upon. My main issues with this approach are:

1. I need a unique counter for each test and I need to remember which counters are already taken
2. It couples my tests to global state
3. If I am not careful I may very well loose the ability to run my tests concurrently, as shown in the tutorial about ETS tables.

I will give it to you that your solution does technically work and that it may end up being less complex than the process approach. So it is something to have in mind. Thanks for sharing!

No, you do not need to, as you are passing counter, which is unnamed and identified by reference.

Like most mocking libraries, fakes, etc. It just mean that sometimes it is just hidden from you. And if you want, you can always use Agent instead of :counters.

Not exactly, just use unnamed ETS tables, and you will be good to go.

So I don’t have to keep track? Quite interesting.

A fair argument. Truth is as long as I can run my tests with async: true I don’t really care.

Like the one :counters create?

You have overall good arguments and have convinced me.

Do you have any links or documentation to :counters.new ?
I check the official erlang docs but was unable to find anything.