Fl4m3Ph03n1x
Strategies to manipulate test spies and stubs?
Background
Recently I have thrown again into the see of testing my functional code with Elixir. However I am finding it rather difficult to create tests for basic manager / collaborators communication because I find Elixir’s toolset lacking when it comes to spies and stubs. Pure functions only whose dependencies get injected, no mocks here, no modules.
Objective
Several approaches have been suggested to me when testing stubs and spies. The objective of this discussion is to have a (somewhat) comprehensive lists of strategies used with their pros and cons.
You may not 100% agree with the definitions I am about to suggest, but for the technical purposes of this discussion let’s simply go with them (you may disagree with me in another discussion
)
Spies
A spy’s main objective is to tell how information about a function, such as:
- Was a function invoked?
- How many times it was invoked
- With what parameters it was invoked
Stubs
A stub’s main purpose is to direct the workflow of your program. This basically boils down to:
- returning an output you define at test level
You may force a function to raise an error or return a given value.
Common approaches
Following I discuss some common approaches to testing.
test process sends messages to self()
pros
- check a given function was called
- check with which arguments it was called
- does not use global state
cons
- cannot tell you a function was invoked precisely X times
- cannot change return value based on how many times a function was invoked
usage of :counters (requires erlang 21.2)
pros
- allows you to know exactly how many times a given function was invoked
- allows you to, based on the number of times a function was called, return different results
- does not use global state
cons
- doesn’t allow you to know with which parameters a function was called with
More?
There are more solutions out there. @peerreynders suggested the use of named ETS tables to store state in tests and @hauleth the use of Agents. I look forward to having them join this conversation (if they want to ofc).
What other strategies do you know?
Most Liked
axelson
we use Mox extensively for our unit testing (although perhaps you disagree that we are unit testing). I tend to agree with @LostKobrakai that making the multiple implementations of a function/module explicit via a behaviour benefits the code by defining an explicit contract.
Also in case you’re not aware with Mox it is very easy for the mocked module to return different values when it is called a second or third time:
test "email sending is handled" do
MyApp.Test.MockEmailSender
|> Mox.expect(:send_email, fn _email_text -> {:ok, :sent} end)
|> Mox.expect(:send_email, fn _email_text -> {:error, :remote_server_offline} end)
# When this calls `MockEmailSender.send_email/1` it will return `{:ok, :sent}`
MyApp.notify_user("hi")
# When this calls `MockEmailSender.send_email/1` it will return `{:error, :remote_server_offline}`
MyApp.notify_user("bye")
end
Also that test above works perfectly with async: true even if you’re running code with Tasks (as long as you’re using Elixir 1.8+).
Sometimes I also stub a Mock to run the actual code. That looks like:
Mox.stub(MyApp.Test.MockEmailSender, :send_email, &MyApp.EmailSender.send_email/1)`
One last thought, I’ve found that using a Mock with an explicit contract lines up very well with the “functional core, imperative shell” school of thought.
LostKobrakai
But you also introducted the idea of injected dependencies, which need to be set somewhere. And that somewhere is most often some global entity, unless you can inject from within the test itself. If you indeed can inject from the test then you should be just fine with e.g. mox Mox — Mox v0.5.0 even if multiple processes (concurrent testing) are involved.
Edit: The difficult part here is how to let :get_fn be aware of where to send data like “I was called with args”, so that the test can check those. If you can directly inject e.g. the pid of the test (and each test has it’s own pid) then the rest is just implementation of sending the correct data back.
hauleth
It depends on the my_lib and there is few possible approaches to that:
Mock my_lib
You can use for example mockery in form of:
def my_request(val) do
mockable(MyLib).get("some_website/val")
end
And then in your test you can use:
test "calls MyLib.get/1" do
mock MyLib, [get: 1], fn _data -> System.unique_integer() end
Subject.my_request(10)
assert_called MyLib, get: 1
end
Or if you want sequence of numbers
test "calls MyLib.get/1" do
counter = :counters.new(1, [])
mock MyLib, [get: 1], fn _data ->
:ok = :counters.add(counter, 1, 1)
:counters.get(counter, 1)
end
Subject.my_request(10)
assert_called MyLib, get: 1
end
Or if you are using Erlang <21.2
test "calls MyLib.get/1" do
pid = start_supervised!({Agent, 0})
mock MyLib, [get: 1], fn _data ->
Agent.get_and_update(pid, &{&1 + 1, &1 + 1})
end
Subject.my_request(10)
assert_called MyLib, get: 1
end
Mock HTTP client
Assume that my_lib use Tesla library for requests then you can use:
setup do
Application.put_env(:tesla, MyLib.HTTPClient, adapter: Tesla.Mock
end
test "API does request to `http://example.com`" do
mock fn ->
%{method: :get, url: "http://example.com/hello"} ->
%Tesla.Env{status: 200, body: "#{System.unique_integer()}"}
end
Subject.my_request(10)
end
Mock target service
If target URL is configurable then you can use solution like Bypass to create fake target of the request instead of mocking MyLib or HTTP client. I used this solution for testing S3 backends in my projects.
So as you can see, there is plenty of possible solutions, and Elixir by default do not force one approach over another, which IMHO is good thing.
Popular in Discussions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









