Using Hammox as a spy?

Suppose I have a code like this:

defmodule Spy do
  @analytics Application.compile_env!(:spy, :analytics)

  def create_article(params) do
    case insert_article(params) do
      {:ok, article} ->
        @analytics.record(:article_created, article)
        if params.published, do: @analytics.record(:article_published, article)
      
      {:error, error} -> {:error, error}
    end
  end

  defp insert_article(params) do
    id = :rand.uniform(1000)
    {:ok, Map.put(params, :id, id)}
  end
end

A typical tests for it would look like this:

defmodule SpyTest do
  use ExUnit.Case
  import Hammox

  defmock(AnalyticsMock, for: Analytics)

  test "old approach: not published" do
    expect(AnalyticsMock, :record, fn :article_created, payload ->
      assert payload.title == "Test"
      :ok
    end)

    Spy.create_article(%{title: "Test", published: false})
  end

  test "old approach: published" do
    expect(AnalyticsMock, :record, fn :article_created, payload ->
      assert payload.title == "Test"
      :ok
    end)

    expect(AnalyticsMock, :record, fn :article_published, payload ->
      assert payload.title == "Test"
      :ok
    end)

    Spy.create_article(%{title: "Test", published: true})
  end
end

While this of course works, it provides a couple disadvantages in my opinion:

  • You don’t follow “arrange-act-assert”, as you actually assert at the beginning
  • In case of testing for publish, you have to add both expectations in a single test case, otherwise you’d get an error.

As a result, I’ve been thinking about bending Hammox to my will a bit and force it to act more like a spy, where I record the interactions somehow and assert about them in the end. Of course, it should ideally work with async tests.

I came up with something like this:

defmodule SpyTest do
  use ExUnit.Case
  import Hammox

  defmock(AnalyticsMock, for: Analytics)

  defmodule AnalyticsStub do
    @behaviour Analytics
    def record(event, payload) do
      send(self(), {:analytics, event, payload})
    end
  end

  describe "new approach" do
    setup do
      stub_with(AnalyticsMock, AnalyticsStub)
      :ok
    end

    test "not published" do
      Spy.create_article(%{title: "Test", published: false})
      assert_received({:analytics, :article_created, payload})
      assert payload.id > 0
    end

    test "published - record creation" do
      Spy.create_article(%{title: "Test", published: true})
      assert_received({:analytics, :article_created, _})
    end

    test "published - record publish" do
      Spy.create_article(%{title: "Test", published: true})
      assert_received({:analytics, :article_published, _})
    end
  end
end

To me it reads much better and I wonder: is someone using an approach like this already? Or maybe there’s a different tool for that? If not, could it be a terrible idea for some reason I don’t yet see?

It might need some setup to ensure the process inbox is empty before running the test, but it is doable and aside from that?

That sounds like trace based testing - Testing System/Unit Behaviour Using Traces/Logs/Signals

The example might be easier to test if its decoupled and the create_article just publishes an event {:article_created, article} that your analytics stuff (or your tests) can subscribe to.

1 Like

Yes, I read that topic later tonight, thinking that it might be what I’m after :wink:

As for publishing event - sure, but I’d want to “spy” on this event bus to check what was pushed to it anyway. So it’s in a way the same problem, but with a different module.