How to describe many contexts in ExUnit without a hierarchy

chrisalley

ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe blocks are forbidden, and instead developers should “build on top of named setups”. The following example used:

defmodule UserManagementTest do
  use ExUnit.Case, async: true

  describe "when user is logged in and is an admin" do
    setup [:log_user_in, :set_type_to_admin]

    test ...
  end

  describe "when user is logged in and is a manager" do
    setup [:log_user_in, :set_type_to_manager]

    test ...
  end

  defp log_user_in(context) do
    # ...
  end
end

For comparsion, here’s how I might lay out the same test using RSpec:

RSpec.describe UserManagement, type: :model do
  describe "some method" do
    context "when user is logged in" do
      before do
        ..
      end

      context "when user is an admin" do
        before do
          ...
        end

        it ...
      end

      context "when user is a manager" do
         before do
           ...
         end

         it ...
      end
    end
  end
end

The reasoning for ExUnit way is solid - it’s easier to glance at a single describe block in isolation and understand exactly what is going on. However, coming from RSpec, the ability the lay out multiple contexts as described above is useful for covering all scenarios. In particular, if there’s three or four different contexts that need to be checked, including different combinations of each, having a hierarchy can help to lay out all of the different pathways through the code. If the contexts are keep flat, we would end up with very long describe/context strings, e.g. “when user is logged in and user is an admin and user is an author and user is a publisher”.

Since ExUnit doesn’t support nested describe/context blocks, what are some alternative strategies to describe these more complex sets of contexts?

4 comments

#testing #exunit

8 17527 4

2017-03-25 17:02:28 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

georgeguimaraes

What do you think about breaking each context (logged in) in its own file?

Post #1

josevalim

Creator of Elixir

Since ExUnit doesn’t support nested describe/context blocks, what are some alternative strategies to describe these more complex sets of contexts?

Your first approach is the alternative. Sometimes we get too hung up on reducing duplication and we forget the downsides of trying to “unify” everything, such as coupling and loss of context. ExUnit wants you to define everything in two lines (the describe and setup) because the next person reading the file will have a better understanding of what is happening.

Go back to a project that uses nested describes/context and try to find out what a single test is doing. This is going to be the process:

You will find the test and try to find the outer most describe
Put “user is logged in” (the outermost describe) in your brain stack
Then find and put “and is a manager” in your brain stack
Then find and put “with multiple accounts” in your brain stack
Remove “with multiple accounts” from your brain stack (because that context has ended without reaching your test)
Then find and put “with a single account” in your brain stack
Found the test

It is such a hassle.

The describe is there to describe what your test is doing. Be descriptive.

Post #2

chrisalley

Thanks for the detailed response, @josevalim. Having dealt with some complex nested specs, I can certainly relate to the desire for decoupling and explicitness when examining individual tests.

My main concern is how to structure contexts without a literal tree of code. But as @georgeguimaraes suggested, the contexts could be placed into seperate files. This would in practice create a kind of hierarchy, with the folder containing the context files acting as the parent. If required, further subfolders could be created to represent grandparent contexts, creating a tree of folders and files instead of a tree of code. Each test would still have the full description of the context in the describe statement so that the test can be understood in isolation, while the folder structure would represent the structure of the many pathways through the code. I imagine a lot of RSpec tests could benefit from this kind of restructuring too, rather than being placed in one giant file that covers every context that the class being tested can be in.

Post #3

tjstankus

Having many pathways through your code is complicating your tests. If you’re going for full test coverage, your tests end up as the Cartesian product of the branches in your code, requiring new test cases exponentially as you add branches. The underlying solution is not to hone in on a way to nest your tests, but to able to test contexts independently. In Ruby and Elixir this is achieved through polymorphism. To use your example, you would test admin behavior and manager behavior independently and treat user polymorphically in code that depends upon user.

Post #4

— All posts loaded —