Scannable map or struct assertions

[marick]

TL;DR: I introduce assert_fields and assert_copy and provide their code. Original blog post here.

---

I used to say “All the words in a test should be about the purpose of the test.” I’ll probably be exploring some of the ramifications of that slogan throughout the blog. For now, I want to focus on a variant:

All the words I look at in a test should be about my purpose for looking at it.

The “I look at” is because of my new emphasis on scannability. Recall from the previous post that I believe tests should help the reader whose eyes are darting from place to place within a test, searching for an answer to a specific question.

Here are some assertions that improve the scannability of tests involving structs or maps.

assert_fields

Consider code like this:

animal = AnimalT.update_for_success(original.id, params)
assert animal.name == "New Bossie"
assert animal.lock_version == 2

A while ago, Steve Freeman and I were pairing, and he reacted badly to code like that. In response, I created an assert_fields function that allows the following:

AnimalT.update_for_success(original.id, params)
|> assert_fields(name: "New Bossie", lock_version: 2)

In addition to chaining the assertion (as in the previous post), I like the way syntax highlighting makes the necessary-but-not-enlightening use of assert_fields fade into the background.

assert_copy

The function tested above produces an updated version of a struct with three kinds of keys:

• keys that should have been left alone,
• … keys whose changed value needs to be checked …
• … and keys whose new value (if any) should be ignored.

A new function, assert_copy, works with assert_fields to handle all three cases in a terse way:

AnimalT.update_for_success(original.id, params)
|> assert_copy(original,
      except:   [name: "New Bossie", lock_version: 2],
      ignoring: [:updated_at])

In the above, :updated_at is the single field whose new value I don’t care about. Perhaps that’s not right. Perhaps I want to make sure that :updated_at has been increased from its original value. I can do that with…

predicates

Actually, I won’t write an assertion for :updated_at. It only has a one-second granularity, and I don’t want to sleep during tests. Anyway, :updated_at is set by the Ecto machinery, so I’ll believe it’s correct if other fields have been changed.

So I’ll make up an example. It’s a test for a bossify function where I require the :tags field to be empty (but I don’t care what kind of Enum it is):

test "sample" do
  bossify("Bossy")
  |> assert_fields(name: "Bossy",
                   tags: &Enum.empty?/1)
end

Fortunately, Elixir functions generally inspect nicely, so an assertion’s failure message can be nice too:

You can also use predicates in the :except arguments to assert_copy.

Source

The version as of this writing is here. There are some features not documented in this post.

[devonestes]

Interesting! I actually have a library that I think would suit this case for you (and hopefully provide really great error messages for you as well). Check it out here: https://hex.pm/packages/assertions

Is there something in your functions that you can’t easily do with that library? If so, I’d love to add it there. I envision that library as (eventually) a big collection of common assertion abstractions like these ones you mention.

[Schultzer]

@marick

nimal = AnimalT.update_for_success(original.id, params

I hope you are aware that you can leverage pattern matching here, so you could simply do all of this in one line.

assert %Animal{name: "New Bossie", lock_version: 2} = AnimalT.update_for_success(original.id, params)

[hauleth]

@marick

Consider code like this:

animal = AnimalT.update_for_success(original.id, params)
assert animal.name == "New Bossie"
assert animal.lock_version == 2

I find it easier to read something like that:

assert %{
    name: "New Bossie",
    lock_version: 2
  } = AnimalT.update_for_success(original.id, params)

This also makes “predicates” simpler:

assert %{
    name: "Bossy",
    tags: tags
  } = bossify("Bossy")
assert Enum.empty?(tags)

[marick]

I use your library, but I didn’t see functions that did the same things. Did I miss something?

I was planning to ask you if you’d accept these functions into your library, so YES to “I’d love to add it there”. How do you feel about chaining assertions (from https://www.crustofcode.com/chained-assertions/)? With the exception of things like assert_raise, the return value from an assert_* is meaningless, so it does no harm to provide return values that allow chaining.

I also have a set of Changeset-oriented assertions. I don’t know how they’d fit into a general-purpose library.

[marick]

Yes. My bias as a person raised in a language culture that reads left-to-right, and so thinks of time as flowing from left to right or top to bottom is that it’s better to put the code that produces the value-to-be-tested before the code that checks that value.

In my Clojure testing library, I illustrated that with book examples

I wouldn’t force that bias on anyone, but for those that share it, this code might be nice.

[marick]

Because I have an unusually bad short-term memory for a programmer, I prefer not to introduce a token (like tags) that I have to remember as I scan from the point of definition to the point of use:

assert %{
    name: "Bossy",
    tags: tags
  } = bossify("Bossy")
assert Enum.empty?(tags)

I’d rather combine the two:

assert %{
    name: "Bossy",
    tags: &Enum.empty?/1
  } = bossify("Bossy")

… or use my preferred notation, which puts the cause visually before the effect (given that one reads in English order).

[devonestes]

assertions isn’t intended to be general purpose - I’m hoping for it to just include everything that is commonly used. Changeset & Repo based assertions are something I’ve been meaning to add for a while, but I haven’t yet been able to come up with an API for those that I like. If you have some ideas for that, definitely open up an issue and we can discuss it there!

I don’t think I’d accept assert_fields since it is an exact copy of asserting on a match but without the really helpful error message (including the colored diff of match failures). I would say that your assert_copy is equivilant to assert_maps_equal or assert_structs_equal, but I like the addition of an except option, since that exact thing would be kind of difficult to do at the moment if you had a map with 30 keys and you wanted to make sure 29 of them are equal, you’d need to list all 29 keys instead of just listing the one that should be allowed to be different.

For the predicates, I made it so all of the functions like assert_maps_equal were composable, so you can do just about anything you like in there. But, for your specific case where you’re asserting on an empty list, I’d prefer to match on the empty list both for clarity (in my book that’s more clear) and also for performance.

[hauleth]

@marick

I’d rather combine the two:

assert %{
    name: "Bossy",
    tags: &Enum.empty?/1
  } = bossify("Bossy")

This has completely different meaning as:

map = %{
  name: "Bossy",
  tags: &Enum.empty/1
}

assert %{
    name: "Bossy",
    tags: &Enum.empty/1
  } = map

Would fail. This would be irritating even more, at least for me.

I like simplicity of current mappings and the fact that these allows me to compose them easily. I never had problems with remembering that few assignments left.

However as @devonestes said, the assert_copy seems interesting though.

[Schultzer]

I’m conflicted here, it’s the assert_copy I don’t get at all. Even the name is confusing me.

Given that this function only returns an approximation (We don’t account for time in our assertion) for a given input.

Maybe this just boils down to the unfamilirity of a strongly type language and how variable assignments works.

[OvermindDL1]

What about an assert_match, with guards and all, it would handle that case fine.

[Schultzer]

I don’t believe these functions are even needed.

The language is explicit and I feel when you get familiar with it you gonna realize you can do a lot more with a lot less.

The benefit of a = b, where a is the approximation of the return value of function x is more scannable (for me) given the roots in the mathematical notation, rather than you have to figure out what function y is asserting.

[hauleth]

assert_copy has wrong name, but the idea is simple, it is equivalent to:

match_keys = Enum.sort(Map.keys(match))
original_keys = Enum.sort(Map.keys(original))

assert a_keys == b_keys

for key <- match_keys -- (Keyword.keys(except) ++ ignoring) do
  assert Map.fetch!(match, key) == Map.fetch!(original, key)
end

for {key, value} <- except do
  assert Map.fetch!(match, key) == value
end

In other words - it checks whether the match has the same structure and fields as origin ignoring ignoring fields and ensuring that except fields are matched exactly as is. This is useful for example when testing functions that return data from DB after updates, to check that only thing that has changed are the provided fields and no other field.

But yes, naming is poor.

[marick]

@devonestes It’s true that the output of assert_fields doesn’t get the coloring, so you can see this:

I’ve thought of hooking into the differencing machinery, but honestly what I’d do is add an assert:

and type ^C-,-a in Emacs to rerun the single test and see:

Sort of a philosophical thing: when an unexpected failure pops you into a test and corresponding code, it’s no reflection on either if you have to spend a minute or two tweaking it to reveal more information.

(Aside: I’ve long noticed that people have different standards for tests and code. For example, people feel that changing tests is somehow more burdensome than changing code. And adding the above seems wronger than sticking an IO.inspect into the failing code, even though they’re really the same sort of thing.

(I feel the same way, but it’s one of the many gut reactions I’ve trained myself out of.)

[marick]

Regarding using predicates in assert_fields:

I took that from my Clojure testing library. People seem to like it. I’m guessing that they do what I do: hardly ever compare two functions for equality. And when they do, they do the equivalent of:

assert struct1.f == struct2.f

After all, a notion of “extended equality” doesn’t prevent anyone from using plain equality.

[marick]

I don’t like the assert_copy name either, but I haven’t been able to think of a better one. assert_match, for example, says “regular expression” to me.

I grant that the name assumes before-and-after comparisons of the “same” data. But Repo.update is far from an unusual function: lots of them transform input structures into output structures.

assert_approximately is maybe better, but it’s awfully long.

[marick]

Meta:

I was an early adopter of Ruby. In one of the early RubyConfs, maybe the first one, Matz said:

Perl has many, many ways to do something.
Python has One True Way.
Ruby has more than one way, but some are better than others.

An oddity is that what’s better can depend on circumstances, which is why providing more than one way is better.

I think pattern matching is the bee’s knees. But it can lead to code that isn’t easily to grasp at a glance, at least for some people.

For example, I had tests that had to dissect changesets to look at subcomponents. I find assertions like this more easy to grasp:

I think it’s important to note that:

1. people who prefer a pattern-matching style can still use it.
2. those people can easily understand assert_fields style.

I think it’s OK to offer people options and let them decide, even if they decide differently than I would.