Run the same test suite over multiple setups

Questions / Help
,
1

I am looking for a smart way to reuse tests.

My project Raxx is a common interface for web servers. So far I have created adapter implementations for two servers cowboy and elli. I am now working an a third adapter for Ace and want to reuse some of my test suits.

For example I have to test the elli adapter.

# test/raxx/adapters/elli/request_test.exs
defmodule Raxx.Elli.RequestTest do
  use ExUnit.Case, async: true

  setup do
    {:ok, _pid} = :elli.start_link [
      callback: Raxx.Adapters.Elli.Handler,
      callback_args: {Raxx.TestSupport.Forwarder, %{target: self()}},
      port: 2020]
    {:ok, %{port: 2020}}
  end

  test "request shows correct method", %{port: port} do
    {:ok, _resp} = HTTPoison.get("localhost:#{port}")
    assert_receive %{method: "GET"}
  end

  # more tests
end

and similar for the cowboy adapter.

defmodule Raxx.Cowboy.RequestTest do
  use ExUnit.Case, async: true

  setup %{case: case, test: test} do
    name = {case, test}
    routes = [
      {:_, Raxx.Adapters.Cowboy.Handler, {Raxx.TestSupport.Forwarder, %{target: self()}}}
    ]
    dispatch = :cowboy_router.compile([{:_, routes}])
    env = [dispatch: dispatch]
    {:ok, _pid} = :cowboy.start_http(name, 2, [port: 0], [env: env])
    {:ok, %{port: :ranch.get_port(name)}}
  end

  test "request shows correct host", %{port: port} do
    {:ok, _resp} = HTTPoison.get("localhost:#{port}")
    assert_receive %{host: "localhost"}
  end
end

The test suit will be the same for all adapters and only the setup code will vary between them. Is there a smart way to reuse the test suits for different setups?

3 Likes
2

Take a look at how ecto does the adapter tests. The gist is:

  1. Common tests in integration_tests/{adapter}/ with a structure similar to what would be normally in test/.
  2. Depending on which adapter you want to test, you change the test_paths config option
  3. In each adapter tests use Code.require_file/2 to load and run the appropriate common tests.
1 Like
3

Yes. I had a similar need recently and ExUnit.CaseTemplate is your friend.

They don’t say this in the docs, but I’ve experimented with this, and it turns out you can also put actual test cases into the template and put the setup code outside the template (i.e. vary it for each different implementation that you create).

So I have a case template file that looks something like this:

defmodule MyApp.MumbleStrategyCase do
  @moduledoc """
  Validates that a MumbleStrategy implementation fulfills
  the basic contract correctly.

  The following pattern will include a set of tests that verifies the MumbleStrategy
  contract:

defmodule MyMumbleStrategyTest do
use MyApp.MumbleStrategyCase

setup do
  {:ok, mumbler} = # create a mumbler instance
  {:ok, mumbler: mumbler}
end

end

"""
use ExUnit.CaseTemplate

using do
  quote location: :keep do

    test "mumbles correctly", %{mumbler: mumbler} do
      assert mumbler.whatever(...)
    end
  end
end
end
2 Likes
4

I guess it is not in the documentation because you have to wrap it in that using block. However even with that it is a nice solution.

5

Thanks @scouten it took a little bit of tweaking but now have my interface tests against three different webservers, exactly what i needed.

1 Like