Stubr - an amazingly stub-only framework for Elixir

Stubr

In functional languages you should write pure functions. However, sometimes we need functions to call external API’s. But these affect the state of the system. So these functions are impure. In non-functional languages you create mocks to test expectations. For example, you might create a mock of a repository. And the test checks it calls the update function. You are testing a side effect. This is something you should avoid in functional languages.

Instead of mocks we should use stubs. Mocking frameworks tend to treat them as interchangeable. This makes it hard to tell them apart. So it is good to have a simple definition. Quoting Martin Fowler:

  • Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it ‘sent’, or maybe only how many messages it ‘sent’.
  • Mocks are objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

So what does Stubr provide:

  • Stubr is not a mock framework
  • Stubr is not a macro
  • Stubr provides canned answers to calls made during a test
  • Stubr makes it easy to create stubs
  • Stubr makes sure the module you stub HAS the function you want to stub
  • Stubr stubs as many functions and patterns as you want
  • Stubr works without an explicit module. You set it up how you want
  • Stubr lets you do asynchronous tests
  • Stubr won’t redefine your modules!
  • Stubr has ZERO dependencies

Example - Adapter for JSON PlaceHolder API

This is a simple JSONPlaceHolderAdapter built using TDD:

defmodule Post do
  defstruct [:title, :body, :userId, :id]
end

defmodule JSONPlaceHolderAdapter do
  @posts_url "http://jsonplaceholder.typicode.com/posts"

  def get_post(id, http_client \\ HTTPoison) do
    "#{@posts_url}/#{id}"
    |> http_client.get
    |> handle_response
  end

  defp handle_response({:ok, %HTTPoison.Response{body: body, status_code: 200}}) do
    post = body
    |> Poison.decode!(as: %Post{})
    {:ok, post}
  end

  defp handle_response({:ok, _}) do
    {:error, "Bad request"}
  end

  defp handle_response({:error, _}) do
    {:error, "Something went wrong"}
  end
end

The injected http_client argument of JSONPlaceHolderAdapter.get/1_post defaults to HTTPoison. This is so we can create a stub using Stubr.

Stubr is good for using test data to define stubs. Pattern match on test data to create the function representations. Use Stubr to create the stub:


@post_url "http://jsonplaceholder.typicode.com/posts"

@good_test_data [
  %{
    id: 1,
    post_url: "#{@post_url}/1",
    expected: %Post{title: "A title", body: "Some body", userId: 2, id: 1},
    canned_body: "{\"userId\": 2,\"id\": 1,\"title\": \"A title\",\"body\": \"Some body\"}"
  },
  %{
    id: 2,
    post_url: "#{@post_url}/2",
    expected: %Post{title: "Another title", body: "Some other body", userId: 3, id: 2},
    canned_body: "{\"userId\": 3,\"id\": 2,\"title\": \"Another title\",\"body\": \"Some other body\"}"
  }
]

test "If the call to get a post is successful, then a return post struct with id, userId, body and title" do
  
  # Dynamically build up the functions to stub. Alternatively, store them with the test data
  functions = @good_test_data
  |> Enum.map(
    fn %{post_url: post_url, canned_body: canned_body}
    # pin operator ^ is used to guarantee the correct response is returned
    -> {:get, fn(^post_url) -> {:ok, %HTTPoison.Response{body: canned_body, status_code: 200}} end}
    end
  )

  http_client_stub = Stubr.stub(HTTPoison, functions)

  for %{id: id, expected: expected} <- @good_test_data do
    assert JSONPlaceHolderAdapter.get_post(id, http_client_stub) == {:ok, expected}
  end
end

The unsuccessful tests work in much the same way:

@bad_test_data [
  %{id: 1, post_url: "#{@post_url}/1", status_code: 400},
  %{id: 2, post_url: "#{@post_url}/2", status_code: 500},
  %{id: 3, post_url: "#{@post_url}/3", status_code: 503}
]

test "If the response returns an invalid status code, then return error and a message" do
  functions = @bad_test_data
  |> Enum.map(
    fn %{status_code: status_code, post_url: post_url}
      -> {:get, fn(^post_url) -> {:ok, %HTTPoison.Response{status_code: status_code}} end}
    end
  )

  http_client_stub = Stubr.stub(HTTPoison, functions)

  for %{id: id} <- @bad_test_data do
    assert JSONPlaceHolderAdapter.get_post(id, http_client_stub) == {:error, "Bad request"}
  end
end

test "If attempt to get data was unsuccessful, then return error and a message" do
  bad_response = {:get, fn(_) -> {:error, %HTTPoison.Error{}} end}

  http_client_stub = Stubr.stub(HTTPoison, [bad_response])

  assert JSONPlaceHolderAdapter.get_post(2, http_client_stub) == {:error, "Something went wrong"}
end

Example - Creating Complex Stubs

Stubr can create complex stubs:

stubbed = Stubr.stub([
  {:gravitational_acceleration, fn(:earth) -> 9.8 end},
  {:gravitational_acceleration, fn(:mars) -> 3.7 end},
  {:gravitational_acceleration, fn(:earth, :amsterdam) -> 9.813 end},
  {:gravitational_acceleration, fn(:earth, :havana) -> 9.788  end},
  {:gravitational_attraction, fn(m1, m2, r) -> Float.round(6.674e-11 *(m1 * m2) / (r * r), 3) end}
])

assert stubbed.gravitational_acceleration(:earth) == 9.8
assert stubbed.gravitational_acceleration(:mars) == 3.7
assert stubbed.gravitational_acceleration(:earth, :amsterdam) == 9.813
assert stubbed.gravitational_acceleration(:earth, :havana) == 9.788
assert stubbed.gravitational_attraction(5.97e24, 1.99e30, 1.5e11) == 3.523960986666667e22

Links

To see how stubs can be used in TDD, see https://www.infoq.com/presentations/mock-fsharp-tdd

Hex: https://hex.pm/packages/stubr

GitHub: https://github.com/leighshepperson/stubr

6 Likes

I suggest you be more ambitious.

In my Midje testing framework for Clojure, I take an approach that I think reveals that the original idea of mocks (it’s about relationships between objects) is quite applicable to functional languages (it’s about relationships between functions).

That mocks are about nailing down relationships is less understood than it should be.

The basic idea starts from this:

  1. Tests are not universally-quantified (“for all”) claims about programs, since they don’t show that something is true for all inputs.
  2. But it’s still useful to declare facts about specific inputs: assert sqrt(4) == 2
  3. Really? Why is it useful?
  4. Because humans can, given enough such facts, decide that the universal statement (forall x, sqrt(x) * sqrt(x) == x) is a safe enough bet.

Thus, tests are “proof-like”. And how do proofs work? They depend on lemmas. That is, the truth of a proof depends on the truth of the lemmas used in it.

Translating into functions, the same can be said: a function’s correctness depends on the correctness of the functions it calls.

In the “give me an example” world of testing, that suggests writing tests of the form "assert f(8) == 3 PROVIDED f calls g with value 8 AND assert g(8) == 2. That decomposes the problem of the correctness of f into two problems:

  1. If g is correct, does f give the right answer?
  2. Is g correct?

If you practice TDD or what’s called in Structure and Interpretation of Computer Programming “programming by wishful thinking”, that lets you work on f now and defer the correctness - and even actual existence - of g until later.

This turns out to be really nice in practice. I find myself missing it in Elixir.

I do grant that most tests don’t need the distinction between “g must be called and…” and “if g is called then [same thing]”. But I think restricting the idea of mocks/lemmas to the state-changing boundaries of a system is too prescriptive.

2 Likes

I like it.

Quick Q. What happens for functions you have not mocked on the original module. does it fall back to using them?

Thanks for your feedback

To some “degree” you can already do this in Stubr,

In this scenario, g will be passed via a module called input into f as a function parameter. For example:

def f(input \\ real_input) do

end

Then create a stub for g:

input_stub = Stubr.stub([{:g, fn 8 -> 2 end}])

Then “as a developer” you know you will be calling the function g inside function f with the value 8, so you can rewrite f as

def f(input  \\ real_input) do
  2 = input.g(8)
end

Then the test is just

assert f(8) == 3

and the other conditions must hold for this to work.

Admittedly, this pushes more work on the developer to arrange the test this way. But it’s “kind of” in-line with this approach https://www.infoq.com/presentations/mock-fsharp-tdd

However, I am going to be adding meta data to Stubr so the stubbed modules can track inputs. It’s possible that this could tie up nicely to with your suggestion, and I’ll certainly explore it.

That said, I am hesitant to add “was called” functionality into Stubr due to reasons explained in Mark Seemann’s blog post where he talks about the difference between Mocks and Stubs in the context of commands and queries.

So although it is possibly out of scope with Stubr, it’s an interesting approach, and it could form the basis for future work

Thanks for your reply!

At the moment the un-stubbed functions are not deferred to the originals.

However, it’s on the road map to add an Auto Stub feature!

Stubr now has an auto-stub feature:

You can auto-stub modules by setting the auto_stub option to true. In this case, if you have not provided a function to stub, it will defer to the original implementation:

stubbed = Stubr.stub!(Float, [
  {:ceil, fn 0.8 -> :stubbed_return end},
  {:parse, fn _ -> :stubbed_return end},
  {:round, fn(_, 1) -> :stubbed_return end},
  {:round, fn(1, 2) -> :stubbed_return end}
], auto_stub: true)

assert stubbed.ceil(0.8) == :stubbed_return
assert stubbed.parse("0.3") == :stubbed_return
assert stubbed.round(8, 1) == :stubbed_return
assert stubbed.round(1, 2) == :stubbed_return
assert stubbed.round(1.2) == 1
assert stubbed.round(1.324, 2) == 1.32
assert stubbed.ceil(1.2) == 2
assert stubbed.ceil(1.2345, 2) == 1.24
assert stubbed.to_string(2.3) == "2.3"

GitHub

Hex

1 Like