Chained assertions

Chat / Discussions
1

TL;DR: Chain assertions for scannability. Here’s some code to help. Original for this post is on my blog.

I suspect tests are more often scanned than read. When a test unexpectedly fails, I think people often jump to its source. But they don’t read that source sequentially from the top; instead, they start at the failing assertion, with the specific goal of understanding what happened. Their eye darts from the assertion to other parts of the test in order to achieve their goal, quite likely skipping over bits of code (like other assertions) that won’t contribute.

If that’s true, it’s useful to arrange tests to make scanning easier. With that in mind, consider an end-to-end-style Phoenix controller test for successfully creating a password by “redeeming” an email token. The test checks two things:

  1. What the externally-visible result is (what page is shown, what’s put in the session, what important messages are shown to the user).
  2. Whether the user can now log in. (Having two kinds of check in the same test is arguably wrong. Let’s leave that aside.)

Here’s the test:

test "the password is acceptable", %{conn: conn, user: user} do
  conn = action__set_fresh_password(conn, @valid_password, @valid_password)
  assert_logged_in(conn, user, @institution)
  assert_no_token_in_session(conn)

  assert_redirected_home(conn)
  assert_info_flash_has(conn, "You have been logged in")

  assert {:ok, _} = Users.attempt_login(user.auth_id, @valid_password, @institution)
end

I want to say that’s less scannable than it should be. In the multi-argument assertions, the conn is the least important argument; it really just gets in the way. And the important functions (action__set_fresh_password and attempt_login) are indented (by different amounts) to the right, which makes their importance not visually obvious.

I propose this instead:

test "the password is acceptable", %{conn: conn, user: user} do
  action__set_fresh_password(conn, @valid_password, @valid_password)
  |> assert_logged_in(user, @institution)
  |> assert_no_token_in_session

  |> assert_redirected_home
  |> assert_info_flash_has("You have been logged in")

  Users.attempt_login(user.auth_id, @valid_password, @institution)
  |> assert_ok
end

It helps with the above problems:

  1. The assertions that are about the same data structure are clearly grouped together.
  2. The two important functions are clearly visible at the top of the two pipelines. (Notice that I turned the original test’s pattern-match of the Users.attempt_login result into a simple assertion that "the login attempt was some variant of :ok".)

There’s a more subtle benefit, though. It’s really tempting just to use assert throughout a test:

assert redirected_to(conn) == PublicController.path(:index)
assert user_id(conn) == user.id
assert institution(conn) == @institution

But that puts an extra burden on the reader who scans: what does this collection of equalities mean?

Chaining pushes away from this temptation. To use the |> operator, assertions have to be functions that return their first argument. I find that having to write those functions encourages me to think a bit more abstractly and more “intention-revealingly”. I’m more likely to ask why someone should care that the assertion is true and then choose the function name with that in mind.

Because I’m an old Lisper, I wrote a macro to define such assertions, saving myself the unbearable burden of having to explicitly return the first argument. Here’s what such definitions look like:

defchain assert_user_sees(conn, claims) when is_list(claims) do
  for claim <- claims, do: assert_user_sees(conn, claim)
end

defchain assert_user_sees(conn, claim) do
  assert(html_response(conn, 200) =~ claim)
end

The defchain macro can be found here. (Note: that’s the version as of 2019-12-11. Assuming I don’t change its location since then, the trunk version is here.)

1 Like
2

Note: I’m starting a project to share small Elixir programming ideas and supporting code. I’m starting with my own, but I’d love to write up other people’s. You can find me at marick@exampler.com or @marick on Twitter.

1 Like
3

Sometimes you want to assert on an incomplete pattern match while also assigning a value to the scope, how do you handle that?

4

Not sure if I’m answering your question, but I also use pattern matching in what I think is the usual way:

      {:ok, retrieved} = Users.one_token(token.text)
      retrieved
      |> assert....

However, I’ll also write helper functions. For example, updating an animal is somewhat complicated, so there a number of tests that use Repo.update in a similar way, so there’s a function that takes care of the pattern matching:

      AnimalT.update_for_success(original.id, params)
      |> assert_fields(name: "New Bossie", lock_version: 2)
      |> assert_copy(original,
                     except: [:name, :lock_version, :updated_at])

There’s also an update_for_error_changeset:

      AnimalT.update_for_error_changeset(original.id, params)
      |> assert_error(name: "has already been taken")

(Note that I have a number of helper functions that I think help with the readability of tests of maps and changesets. I’ll be writing about those too.)

1 Like
5

I appreciate your focus on “scanability” @marick . That was what ultimately turned me off on Clojure (which I know you have a history with) and I suppose I’ve been lax in applying that to my Elixir code. I appreciate that reminder.

I do still caution against overuse of pipelines, but I’m not sure tests are a risky situation for that. Food for thought.

1 Like