Dealing with tests for projects that do a lot of side-effects

I ended up with my own library rambla (the library named after the street in Barcelona and the name was taken as an allusion to Broadway where Broadway is something one uses to get to the town and Rambla is the opposite, getting data out of a town.)

The main concept is “channels” that might be attached to connections. Channel names mustn’t be unique across connections, and publishing to, say, channel_1 would publish to all the connections (handlers/adapters) having the channel with such a name defined. To define the new connection (handler/adapter,) one should implement Rambla.Handler behaviour, which is already implemented for a bunch of destinations out of the box. Two special handlers, Mock and Stub come with a library.

That architecture allows to use Rambla.publish(:channel_1, %{foo: 42}) and transparently mock/stub it in :test by defining the different set of channels (mock instead of amqp + redis in :prod.)

See testing section in docs for details.

Nice library!

So from what I understand it works on the same pub/sub principle as PubSub? How do you solve the problem when you require synchronous responses from the channel, does that work?

Well, no, AFAICT. Channels are configurable yet write-only beasts, there is no sub part. The existing handlers do not support synchronous responses, AFAIR, but writing a specific handler is a no-brainer. One might copy-paste the existing one into the new handler and add something like

@impl Rambla.Handler
def handle_publish(message, options, conn) do
  ...

  case Map.get(message, :synch) do
    pid when is_pid(pid) -> send(pid, ...)
    _ -> :ok
  end
end 

or, alternatively, one can pass a callback to publish/3 and have all the low-level connection stuff to deal with (not recommended, only as a last resort.)

Imo the most important thing is figuring out what part of the whole piece you are interested in testing. Is it that the correct requests are made? Is it that the statemachine built in the genserver moves between state as expected? Are there some calculations/data transformations that need to be correct? If the answer is “everything” than it might be time to split the problem up into smaller pieces.

Splitting runtime constructs that are expected to be a single unit is not trivial, hence why the question was asked.

Theoretically this could be split in multiple processes, but this also makes error propagation/handing a lot more complicated.

Splitting this into more processes was not what I was suggesting. Again I’m wondering what exactly you’re trying to test, because that would inform the best way of going forward. I can’t see how a multi step process involving multiple external systems couldn’t be broken down into simpler to test parts.

You got a good point.

I think what I am trying to say is that I don’t want to have to define separated mocks to test the bigger functionality.

For example let’s suppose we have the following scenario: (genserver) —calls–> do_some_redis_operation().

Let’s suppose I have already tests written for do_some_redis_operation() with the correct mocks, it seems that I have to define a set of mocks again for the tests of genserver. Arguably, these definitions could be in a single place, however this doesn’t change the fact that I have to define the mocks once again, this makes me very nervous about having to maintain the set of mocks for all these situations.

With a pub/sub system, you basically don’t have to do that as you are separating the parts by events, even though there are lots of downsides by doing this too.

I think I will reconsider using mocks for anything but the http API, as it seems to be creating more problems than it solves.

You’re still only explaining what you do, not what the intent of the test is. If do_some_redis_operation() is tested, what part of (genserver) —calls–> do_some_redis_operation() do you care about?

I’ve been meaning to read this forever. Maybe it’s helpful? James Shore: Testing Without Mocks: A Pattern Language

The main intent of the test is OFC to check that do_some_redis_operation() is called and other behavior that is being done by the genserver.