devonestes

Assertions - helpful assertions to help you write better tests

Introducing assertions, the library that helps you write really great test assertions!

GitHub: GitHub - devonestes/assertions: Helpful assertions for ExUnit · GitHub
Hex.pm: assertions | Hex
HexDocs: Assertions — Assertions v0.22.0

This library aims to wrap up common types of assertions that many applications come across in a nice, reusable, composable fashion, and with really exceptional error messages.

Here’s just one example of one use case - comparing a list of structs for equality.

Imagine you have a Phoenix app, and you need to test that the response from a function is basically equal to some other list of structs that you created earlier. We can’t compare them directly because the order of the structs in the list is not guaranteed, so assert list1 == list2 won’t work. Plus, the structs in our list might have differing assocations preloaded or not, meaning we also can’t reliably do assert user1 == hd(list). So, to test this reliably, we’ll need to do something like this:

defmodule UsersTest do
  use ExUnit.Case, async: true

  describe "update_all/2" do
    test "updates the given users in the database and returns those updated users" do
      alice = Factory.insert(:user, name: "Alice")
      bob = Factory.insert(:user, name: "Bob")

      updated_names = 
        [{alice, %{name: "Alice A."}, {bob, %{name: "Bob B."}}}]
        |> Users.update_all()
        |> Enum.map(& &1.name)

      all_user_names =
        User
        |> Repo.all()
        |> Enum.map(& &1.name)

      Enum.each(["Alice A.", "Bob B."], fn name ->
        assert name in updated_names
        assert name in all_user_names
      end
    end
  end
end

But that leaves a lot to be desired. There’s extra code to pull out just the names from the response and we’re obscuring the function that we’re actually testing here. The call to Users.update_all/1 is kind of buried in there.

Let’s see how we might do this with assertions:

defmodule UsersTest do
  use ExUnit.Case, async: true

  describe "update_all/2" do
    test "updates the given users in the database and returns those updated users" do
      alice = Factory.insert(:user, name: "Alice")
      bob = Factory.insert(:user, name: "Bob")

      result = Users.update_all([{alice, %{name: "Alice A."}, {bob, %{name: "Bob B."}}}

      result
        |> Enum.map(& &1.name)
        |> assert_lists_equal(["Alice A.", "Bob B."])

      assert_lists_equal(result, Users.list_all(), &structs_equal?(&1, &2, [:name]))
    end
  end
end

It’s shorter, easier to change, and much more clear about the function being tested in this test. It also gives us really wonderful error messages, which you can see more examples of in the README on GitHub.

There are many more assertions there, and if there are any other common ones that folks uses often in their projects I’d be happy to add them. I want this to be the one library that has all the common helper functions that anyone needs to write really great tests!

11 comments

#library #assertions

10 12102 11

2019-03-18 08:20:53 UTC

First Post!

axelson

Scenic Core Team

This looks very useful! I think it makes sense to use this as an additional base library and then write/name your own assertions on top of it. For example I use this function extensively in my tests:

  @doc """
  Helper for checking that for two structs, or two lists of structs have the
  same id keys
  """
  def assert_ids_match(list1, list2) when is_list(list1) and is_list(list2) do
    list1_ids =
      list1
      |> Enum.map(fn
        %{id: id} -> id
        nil -> nil
      end)
      |> Enum.sort()

    list2_ids =
      list2
      |> Enum.map(fn
        %{id: id} -> id
        nil -> nil
      end)
      |> Enum.sort()

    assert list1_ids == list2_ids
  end

  def assert_ids_match(%{id: id1}, %{id: id2}) do
    assert id1 == id2
  end

And with your assertions library it would look something more like:

import Assertions

def assert_ids_match(left, right) when is_list(left) and is_list(right) do
  assert_lists_equal(left, right, &assert_ids_match/2)
end

def assert_ids_match(%{id: left_id}, %{id: right_id}) do
  assert left_id == right_id
end

I do have a few quibbles with the naming of a few checks. For example I think that assert!/1 would be better named as assert_true or assert_equals_true to make it more obvious that it is checking that the value is exactly equal true (and not just truth). Although a simple assert expression == true is what I would probably still use.

But what I would really like would be a solution that would let you write a test like (where pin operator is used in the same fashion as ecto queries):

result = some_function(data)
assert_pin_match %{id: ^data.id, name: ^original.name} = result

(I talk about this more at Can you pin a temporary variable while asserting a pattern match)

Post #1

Most Liked

devonestes

The improvements in 1.8 are very helpful, but it doesn’t do the shrinking that I do in assertions, and it also doesn’t expand all variables in the argument to assert. For example, a common way to test unordered list equality is something like this:

list = [1, 2, 3]
assert Enum.all?([2, 1, 4], & &1 in list)

That gives us this failure:

  1) test example (AssertionsTest)
     test/assertions_test.exs:8
     Expected truthy, got false
     code: assert Enum.all?([2, 1, 4], &(&1 in list))
     arguments:

         # 1
         [2, 1, 4]

         # 2
         #Function<6.120201396/1 in AssertionsTest."test example"/1>

     stacktrace:
       test/assertions_test.exs:10: (test)

So we can see the one list, but we can’t see the elements in the other list that we’re checking for unordered equality. And then there’s also the problem of that not actually checking equality, which many people think it does

But if we use assert_lists_equal then we write the test like this:

list = [1, 2, 3]
assert_lists_equal(list, [2, 1, 4])

Which gives us this output:

  1) test example (AssertionsTest)
     test/assertions_test.exs:8
     Comparison of each element failed!
     code:  assert_lists_equal(list, [2, 1, 4])
     arguments:

         # 1
         [1, 2, 3]

         # 2
         [2, 1, 4]

     left:  [3]
     right: [4]
     stacktrace:
       test/assertions_test.exs:10: (test)

So there we can see the differing elements between the two lists in left: and right: , which I find really helpful. Since the standard diffing that we do with something like list1 == list2 assumes that order matters, it doesn’t work when comparing lists where the order doesn’t matter (which is a great deal of the time). I thought the best way to show the diff would be to remove the common elements.

I also do this shrinking when comparing structs/maps with assert_maps_equal/3 - it will just show you what’s different instead of showing you the whole big map, and you still get the colored diff if the maps are nested.

Post #5

josevalim

Creator of Elixir

Beautiful.

Post #6

devonestes

For the pin thing, I get around that by doing:

result = some_function(data)
expected = %{id: data.id, name: orginal.name}
assert_maps_equal(result, expected, Map.keys(expected))

Also, one thing to note is that the function you pass to assert_lists_equal should return true or false, not raise an exception (which is what assert does under the hood), so you’d actually want:

import Assertions

assert_lists_equal(left, right, &ids_match?/2)

def ids_match?(%{id: id}, %{id: id}), do: true
def ids_match?(_, _), do: false

This is because the “shrinking” that happens to give the really great error messages relies on being able to compare every element in one list to every element in the other, and if you’re asserting then it’ll fail if there is any element in one list that isn’t equal to any other element in the other.

However, what I do is have functions like structs_equal? or maps_equal? that take a set of keys to use to determine if the two elements are equal in this scenario, like this:

import Assertions

assert_lists_equal(left, right, &structs_equal?(&1, &2, [:id]))

def structs_equal?(left, right, keys) do
  Enum.all(keys, fn key -> Map.get(left, key) == Map.get(right, key) end)
end

But that’s totally the intention behind all these assertions - to give you the basics that then allows you to use your own comparison functions to determine equality for the situation you’re in. == just doesn’t always cut it, ya know

Post #2