Behavior of setup_all in ExUnit

Questions / Help
,
1

I’ve been playing round with test fixtures using setup_all, and I’m considering opening a pull request which includes a few more concrete examples.

The specific behavior that I found to be a bit of a secret is that your test statements can receive values from the setup_all blocks (!!!). In addition to being a bit of a secret bit of functionality, I also had some confusion as to why the setup_all block could return either a map OR a tuple (with a map), and the test statement receives only the map. This is the type of “magic” or hidden functionality that can make code difficult to understand, so I would like to ask for some insight from the forum members on this topic.

Would adding the following as ## Examples in the documentation be useful?

defmodule ExampleATest do
  use ExUnit.Case

  setup_all do
    %{is_a_map: true}
  end

  test "this function receives the output of the setup_all callback", value do
    assert true == value[:is_a_map]
  end
end

defmodule ExampleBTest do
  use ExUnit.Case

  setup_all do
    [keyword: true]
  end

  test "keyword lists are also allowed", value do
    assert true == value[:keyword]
  end
end

defmodule ExampleCTest do
  use ExUnit.Case

  setup_all do
    {:ok, %{is_a_map: true}}
  end

  test "surprisingly, if an :ok tuple is returned, tests only receive the value", value do
    assert true == value[:is_a_map]
  end
end
2

It is documented here https://hexdocs.pm/ex_unit/ExUnit.Callbacks.html#module-context

But more examples wouldn‘t hurt I guess. Maybe ask first in Elixir repo.

3

Yeah, I read that but I didn’t understand much of it from the examples given. “Your documentation has too many examples” said no developer ever. I’ll put together a PR and see what comes of it. Thanks!

1 Like
4 
defmodule EdnoTest do
  use ExUnit.Case

  setup_all do
    {:ok, setup_all: :rand.uniform(100)}
  end

  setup do
    {:ok, setup: :rand.uniform(100)}
  end

  test "1", context do
    IO.inspect Map.take(context, [:setup_all, :setup])
  end

  test "2", context do
    IO.inspect Map.take(context, [:setup_all, :setup])
  end
end

Running this test suite yields this:

%{setup: 85, setup_all: 6}
.%{setup: 44, setup_all: 6}
.

setup_all has the same value for all tests in the suite – namely it’s called once for the test file (the suite) – while setup is called on each test.

Not trying to be demeaning, just legitimately curious what is tripping you.

6 Likes
5

That’s a great example! Worthy of being included in the docs.

What’s tripping me is that the docs do not have examples as clear as that. I know I am an educator, so my standards for documentation may be high, but I feel the current docs could help reduce mental friction and clarify this feature more quickly by including more examples. Explanations are a nice-to-have; examples are paramount.

Specifically:

  • The docs nowhere mention the notion of a Test Fixture, and that’s really what these functions are relevant to. Other languages/frameworks refer to such functions as fixtures, so the mere act of name-dropping that term will help the light-bulb go off for some people.
  • The examples of the context variable could be more clear (e.g. like yours).
  • I haven’t seen a clear example of how/why you can return a tuple OR a map and the context comes through the same. That’s kind of a weird bit of unseen macro “magic” that is rare in the functional world.

I’m working on the PR to include some clarifications to the existing docs.

4 Likes
6

I did make the mind-mapping from the previous TDD terminology I was familiar with, and connected it to ExUnit myself. But I agree that it should be explicit in the docs.

Sounds like you can use your educator / copywriter skills to help improve the docs then. :slight_smile:

7

I’d guess that it’s just for ease of use and sometimes to make use of functions that return ok/error tuples in other parts of the code.