I was wondering how people usually test projects that have to rely heavily on side-effects.
Imagine I have a genserver that ingests messages from RabbitMQ and has to interact with a mongodb, redis instance and maybe do some http requests afterwards. I currently mock all these interactions, however when writing tests, the amount of mock setup is insane and it is very hard to read, not to mention that it is unclear what is being tested at that point.
I can’t help it but feel that the approach where you mock everything in a single place is wrong, but I cannot understand how you would isolate this functionality, at least without starting to mock internal APIs.
I’ve already used PubSub to make this more manageable in some places, however the problem is that the topology of event passing is not good for places where you want to receive a response back (for example to ACK a AMQP message if processed successfully).
I ended up with my own library rambla (the library named after the street in Barcelona and the name was taken as an allusion to Broadway where Broadway is something one uses to get to the town and Rambla is the opposite, getting data out of a town.)
The main concept is “channels” that might be attached to connections. Channel names mustn’t be unique across connections, and publishing to, say, channel_1 would publish to all the connections (handlers/adapters) having the channel with such a name defined. To define the new connection (handler/adapter,) one should implement Rambla.Handler behaviour, which is already implemented for a bunch of destinations out of the box. Two special handlers, Mock and Stub come with a library.
That architecture allows to use Rambla.publish(:channel_1, %{foo: 42})and transparently mock/stub it in :test by defining the different set of channels (mock instead of amqp + redis in :prod.)
So from what I understand it works on the same pub/sub principle as PubSub? How do you solve the problem when you require synchronous responses from the channel, does that work?
Well, no, AFAICT. Channels are configurable yet write-only beasts, there is no sub part. The existing handlers do not support synchronous responses, AFAIR, but writing a specific handler is a no-brainer. One might copy-paste the existing one into the new handler and add something like
@impl Rambla.Handler
def handle_publish(message, options, conn) do
...
case Map.get(message, :synch) do
pid when is_pid(pid) -> send(pid, ...)
_ -> :ok
end
end
or, alternatively, one can pass a callback to publish/3 and have all the low-level connection stuff to deal with (not recommended, only as a last resort.)
Imo the most important thing is figuring out what part of the whole piece you are interested in testing. Is it that the correct requests are made? Is it that the statemachine built in the genserver moves between state as expected? Are there some calculations/data transformations that need to be correct? If the answer is “everything” than it might be time to split the problem up into smaller pieces.
Splitting this into more processes was not what I was suggesting. Again I’m wondering what exactly you’re trying to test, because that would inform the best way of going forward. I can’t see how a multi step process involving multiple external systems couldn’t be broken down into simpler to test parts.
I think what I am trying to say is that I don’t want to have to define separated mocks to test the bigger functionality.
For example let’s suppose we have the following scenario: (genserver) —calls–> do_some_redis_operation().
Let’s suppose I have already tests written for do_some_redis_operation() with the correct mocks, it seems that I have to define a set of mocks again for the tests of genserver. Arguably, these definitions could be in a single place, however this doesn’t change the fact that I have to define the mocks once again, this makes me very nervous about having to maintain the set of mocks for all these situations.
With a pub/sub system, you basically don’t have to do that as you are separating the parts by events, even though there are lots of downsides by doing this too.
I think I will reconsider using mocks for anything but the http API, as it seems to be creating more problems than it solves.
You’re still only explaining what you do, not what the intent of the test is. If do_some_redis_operation() is tested, what part of (genserver) —calls–> do_some_redis_operation() do you care about?
Could you split the split the statemachine behind this out? Make this more of a state + event → next_state type of deal, where you test the steps it takes individually and therefore with only the mocks needed for a certain step?
Plus maybe a single test to make sure the pieces are then wired together well and if you want to be triple secure you can add some (stateful) property tests to make sure you didn’t overlook a permutation of possible events in sequence.
Damn, this sounds amazing , do you have an example of how something like this would look like? I have used gen_statem before, I think it can be used for this?
This is exactly the topology I am looking for tests, no repeating tests and making sure the subsequent functionality is called.
I’ve also used a pattern in which I separate the complex conditional logic (not technically a state machine in my case) that decides what resulting action to take from the code that performs that side-effect action. I can unit test the core logic without performing any action. Something like:
def high_level_feature(inputs) do
{function, params} = core_complex_logic(inputs)
apply(__MODULE__, function, params)
end
def core_complex_logic(inputs) do
...
{:action_function, [param1, param2]}
end
Tests can focus on core_complex_logic and cover all sorts of corner cases, etc. and validate that the expected two-tuple is returned. It’s fast, and has no mocks. Sure, technically that apply call is not being covered in these tests, however, there’s going to be one or two higher-level smoke tests that would execute high_level_feature.
Not at hand. But as @gregvaughn suggested I’d start with a module and functions on it and see how far you can get without a process involved – leaving the process to few smoke test/integration tests to make sure the plain logic was wired together with the process callbacks correctly.
You don’t need to use apply with this approach. You can do:
def high_level_feature(inputs) do
{action, params} = core_complex_logic(inputs)
case action do
:specific_action -> specific_action(params)
# ...
end
end
def core_complex_logic(inputs) do
# ...
{:specific_action, [param1, param2]}
end
Then the LS should work just fine. This might even be preferable since you can introduce some exhaustiveness checking.
It’d be more verbose of course, but that might be worth the tradeoff. I often like paying the verboseness tax if I’m reimbursed with obvious correctness.
But also, the freedom to have specific_action/1 accept whatever the heck params happens to be is nice. That I could switch from
defp specific_action(param1, param2) do
to
defp specific_action([param1, param2]) do
without batting an eye is cool. The latter is even compatible with guards! Sure the @spec would be less helpful (if I were to bother), but I’m already in defp land so I (personally) don’t care so much.
, do you have an example of how something like this would look like? I have used 
