Fl4m3Ph03n1x

Fl4m3Ph03n1x

Strategies to manipulate test spies and stubs?

Background

Recently I have thrown again into the see of testing my functional code with Elixir. However I am finding it rather difficult to create tests for basic manager / collaborators communication because I find Elixir’s toolset lacking when it comes to spies and stubs. Pure functions only whose dependencies get injected, no mocks here, no modules.

Objective

Several approaches have been suggested to me when testing stubs and spies. The objective of this discussion is to have a (somewhat) comprehensive lists of strategies used with their pros and cons.

You may not 100% agree with the definitions I am about to suggest, but for the technical purposes of this discussion let’s simply go with them (you may disagree with me in another discussion :stuck_out_tongue: )

Spies

A spy’s main objective is to tell how information about a function, such as:

  1. Was a function invoked?
  2. How many times it was invoked
  3. With what parameters it was invoked

Stubs

A stub’s main purpose is to direct the workflow of your program. This basically boils down to:

  1. returning an output you define at test level

You may force a function to raise an error or return a given value.

Common approaches

Following I discuss some common approaches to testing.

test process sends messages to self()

pros

  • check a given function was called
  • check with which arguments it was called
  • does not use global state

cons

  • cannot tell you a function was invoked precisely X times
  • cannot change return value based on how many times a function was invoked

usage of :counters (requires erlang 21.2)

pros

  • allows you to know exactly how many times a given function was invoked
  • allows you to, based on the number of times a function was called, return different results
  • does not use global state

cons

  • doesn’t allow you to know with which parameters a function was called with

More?

There are more solutions out there. @peerreynders suggested the use of named ETS tables to store state in tests and @hauleth the use of Agents. I look forward to having them join this conversation (if they want to ofc).

What other strategies do you know?

Most Liked

axelson

axelson

Scenic Core Team

:raising_hand_man: we use Mox extensively for our unit testing (although perhaps you disagree that we are unit testing). I tend to agree with @LostKobrakai that making the multiple implementations of a function/module explicit via a behaviour benefits the code by defining an explicit contract.

Also in case you’re not aware with Mox it is very easy for the mocked module to return different values when it is called a second or third time:

test "email sending is handled" do
  MyApp.Test.MockEmailSender
  |> Mox.expect(:send_email, fn _email_text -> {:ok, :sent} end)
  |> Mox.expect(:send_email, fn _email_text -> {:error, :remote_server_offline} end)
  
   # When this calls `MockEmailSender.send_email/1` it will return `{:ok, :sent}`
  MyApp.notify_user("hi")

  # When this calls `MockEmailSender.send_email/1` it will return `{:error, :remote_server_offline}`
  MyApp.notify_user("bye")
end

Also that test above works perfectly with async: true even if you’re running code with Tasks (as long as you’re using Elixir 1.8+).

Sometimes I also stub a Mock to run the actual code. That looks like:

Mox.stub(MyApp.Test.MockEmailSender, :send_email, &MyApp.EmailSender.send_email/1)`

One last thought, I’ve found that using a Mock with an explicit contract lines up very well with the “functional core, imperative shell” school of thought.

LostKobrakai

LostKobrakai

But you also introducted the idea of injected dependencies, which need to be set somewhere. And that somewhere is most often some global entity, unless you can inject from within the test itself. If you indeed can inject from the test then you should be just fine with e.g. mox Mox — Mox v0.5.0 even if multiple processes (concurrent testing) are involved.

Edit: The difficult part here is how to let :get_fn be aware of where to send data like “I was called with args”, so that the test can check those. If you can directly inject e.g. the pid of the test (and each test has it’s own pid) then the rest is just implementation of sending the correct data back.

hauleth

hauleth

It depends on the my_lib and there is few possible approaches to that:

Mock my_lib

You can use for example mockery in form of:

def my_request(val) do
  mockable(MyLib).get("some_website/val")
end

And then in your test you can use:

test "calls MyLib.get/1" do
  mock MyLib, [get: 1], fn _data -> System.unique_integer() end

  Subject.my_request(10)

  assert_called MyLib, get: 1
end

Or if you want sequence of numbers

test "calls MyLib.get/1" do
  counter = :counters.new(1, [])

  mock MyLib, [get: 1], fn _data ->
    :ok = :counters.add(counter, 1, 1)
    :counters.get(counter, 1)
  end

  Subject.my_request(10)

  assert_called MyLib, get: 1
end

Or if you are using Erlang <21.2

test "calls MyLib.get/1" do
  pid = start_supervised!({Agent, 0})

  mock MyLib, [get: 1], fn _data ->
    Agent.get_and_update(pid, &{&1 + 1, &1 + 1})
  end

  Subject.my_request(10)

  assert_called MyLib, get: 1
end

Mock HTTP client

Assume that my_lib use Tesla library for requests then you can use:

setup do
  Application.put_env(:tesla, MyLib.HTTPClient, adapter: Tesla.Mock
end

test "API does request to `http://example.com`" do
  mock fn ->
    %{method: :get, url: "http://example.com/hello"} ->
        %Tesla.Env{status: 200, body: "#{System.unique_integer()}"}
  end

  Subject.my_request(10)
end

Mock target service

If target URL is configurable then you can use solution like Bypass to create fake target of the request instead of mocking MyLib or HTTP client. I used this solution for testing S3 backends in my projects.


So as you can see, there is plenty of possible solutions, and Elixir by default do not force one approach over another, which IMHO is good thing.

Where Next?

Popular in Discussions Top

Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
arpan
Hello everyone :wave: Today I am very excited to announce a project that I have been working on for almost 3 months now. The project is...
New
AlexMcConnell
The reason that Rails is as popular as it is is because it’s very easy for relatively inexperienced developers to get a lot of work done....
588 19652 166
New
IVR
Hi all, I’ve seen a number of related threads in the past, but I’d still be very curious to hear an up-to-date opinion on this topic. I...
New
sergio
There’s a new TIOBE index report that came out that shows Elixir is still not in the top 50 used languages. It also goes on to call Elix...
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
chulkilee
Here are the list of HTTP client libraries/wrappers, and some thoughts on HTTP client in general. I’d like to hear from others how they w...
New
joeerl
I’m playing with Elixir - It’s fun. I think @rvirding does give Elixir courses these days. Re: files and database - when I given Erlang ...
New
jesse
Hi everyone, I hesitated to post this here because I don’t want you to think I’m spamming, but I’ve been working on a Platform-as-a-Serv...
New
slashdotdash
Phoenix Live View is now publicly available on GitHub. Here’s Chris McCord’s tweet announcing making it public.
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement