Mocking the same function in 2 separate tests while async: false makes test fail

Hey,

I am using this mocking library:

and I am experience something weird:

I have 2 test cases in 2 different folders in the test folder that mock the same function and then asserts that this function has been called.

with_mock(CoolModule, do_something: fn _ -> :ok end

and then at the end

assert_called(CoolModule.do_something(2))

One of the tests, always the same fails if I run the tests together.

If I run the failing test by itself then it passes, or if I comment out the other test that also makes the test pass.

If I put a sleep in the failing test that also makes it pass.

I have async: false set on the test module.

According to the docs of Mock this should work.

Any ideas why this could be?

Thanks

Hi!

The best advice I can provide here is to, if possible, switch to using behaviours and Mox, following the advice outlined in Mocks and explicit contracts - Dashbit Blog, because this allows to keep async tests, and uses per-process expectations (even if it can be a bit harder to use in some contexts, in my experience), with clear boundaries.

On your specific question, also, without more code it will probably be a bit difficult to provide some help. If you have a way to share a smaller reproduction (and I know it can be hard to do), you will get more chance here! (or a bit more concrete elements).

1 Like

The overall vibe of these symptoms is “there is something leaking from the execution of one test and affecting the other test”, since they go away when run in isolation.

“If I put a sleep in the failing test that also makes it pass” sounds like a race: something that cleans up the first test is running in parallel with the second test, and if the second test beats the cleanup you get a failure.

I don’t see anything obviously asynchronous in the mock source - this issue mentions switching to ExUnit.at_exit instead of after, but the suggestion was not implemented.

2 Likes