For several years, I’ve found integration testing in Elixir to be very hard to do right and very painful. Now I don’t care much for integration testing but people always seem to want to test this way. They want to start a process and do some things to it, and then check for the side effects it causes somewhere else, probably several processes away.

For example, there is a process that handles user chat messages. The process also triggers telemetry based on the messages, and telemetry is captured by another process and queued up to be saved in an analytics database. So the test scenario is: start a chat process, send some messages, and ensure the database contains the correct statistics. I don’t think it is an especially good idea to test this way, but it doesn’t seem entirely unreasonable either.

But how do we do it? Mocking the repo or something near the database insert? Most of the mocking libraries are kind of iffy and can cause problems or don’t work async etc. Then there is Mox, but it only works with behaviors. So I’ve seen people add a @callback to a module, just so it can be mocked, there isn’t even a behaviour. I don’t like that because now we’re creating half of a behaviour just for tests. Additionally you need to put stuff in the Application env, causing clutter. That is two aspects of mocking with Mox that require adding test-only code in the regular (non-test) codebase. So I don’t want to do this because in most cases, other than Mox requiring it, there is no reason for adding a behaviour.

Then there’s the other obvious option which is add a lot of sleeps which is bad for obvious reasons.

Now we reach slightly more esoteric techniques like using :erlang.trace as described in e.g. this post. This is actually quite decent if you can use it. You find a process that is supposed to be called and ensure that it is in fact called, using assert_receive. If your process gets a lot of messages it can take a lot of time to get the right pattern for the assert because you have to fish it out of a very long message inbox printed in the terminal. I guess it’s actually only half-decent. And now the process is supposed to do a database insert. And database processes can’t be traced as easily because there is a pool of them. Back to square one.

Of course people are going to reply with stuff like “well in Javascript and Ruby you can just overwrite anything and that is bad because of reasons” and “in Java you have to have an IoC container and that is bad”. And the obvious “you are doing it wrong” / “just don’t test this way”. All I can say is, I respect and appreciate all the work various people have done to make testing in Elixir possible, and I like ExUnit, but in over 5 different programming languages I’ve used, these kinds of tests are the most painful in Elixir.

I guess it boils down to testing things that happen across processes is inherently hard. That’s why I try to avoid it, only test a single process / module as much as possible. But how do you convince other people to avoid these kinds of tests? In some cases they are not that hard to write, but you pay the price later anyway when you rewrite them and they are no longer easy.