ExUnit and binaries

Questions / Help
,
1

1. ExUnit compares lists by value (only the value not matching is highlighted)

code:  assert [1, 1] = [1, 2]
left:  [1, 1]
           - (red highlight)
right: [1, 2]

Binaries are treated as one value (the whole binary is highlighted)

code:  assert <<1, 1>> = <<1, 2>>
left:  <<1, 1>>
       -------- (red)
right: <<1, 2>>

(this especially confusing when using module attributes in the match.)

would it be possible to write a custom assertion that only highlights the byte not matching?

code:  assert <<1, 1>> = <<1, 2>>
left:  <<1, 1>>
            -
right: <<1, 2>>

2. binaries are always displayed as base-10 …

assert <<0xBE, 0xEF>> = <<0xBE, 0xED>>
...
code:  assert <<190, 239>> = <<190, 237>>
left:  <<190, 239>>
right: <<190, 237>>

… or even worse as strings (if it happens to be all-printable)

assert <<0xBE::16>> = <<66, 69>>
code:  assert <<190::16>> = <<66, 69>>
left:  <<190::16>>
right: "BE"

Is there a way to set the base for assertions? (like in inspect(x, base: :hex))

3. what I dream about at night

the ideal would be if ExUnit would layout the right-side binary like the left side, so if I have this assertion:

assert <<1::1, 0::1, 3::2, 15::4>> = <some expression that results in a binary>

instead of

left:  <<1::1, 0::1, 3::2, 15::4>>
       --------------------------- (red)
right: "?"

I’d get

left:  <<1::1, 0::1, 3::2, 15::4>>
         ----                   
right: <<0::1, 0::1, 3::2, 15::4>>
2 Likes
2

I am no exunit expert at all but I am guessing that a major cause of this is that a binary just contains the raw data, the bytes, and no information of how the binary was created. Which allows you to do some real fun things. e.g. pull apart an IEEE754 float into its separate parts in one go:

iex(4)> << sign :: 1, exponent :: 11, fraction :: 52 >> = << 42.2 :: float >>
<<64, 69, 25, 153, 153, 153, 153, 154>>
iex(5)> {sign, exponent, fraction}
{0, 1028, 1435522381224346}
4 Likes
3

Automatically? No, but you can always do it manually via assert/2 and providing custom message.

4

Sure, I could build a custom message, but it’ll never be as beautiful as the ExUnit diffs.

As it seems ExUnit doesn’t like binaries at all. Look at these binaries, that are just "foo" and "fox"

iex(2)> <<102, 111, 111>>
"foo"
iex(3)> <<102, 111, 120>>
"fox"

When I match foo and fox as strings I get a diff only for o and x.

assert "foo" = "fox"
...
code:  assert "foo" = "fox"
left:  "foo"
          -
right: "fox"

When I match the binaries the whole left binary is marked red.

assert <<102, 111, 111>> = <<102, 111, 120>>
...
code:  assert <<102, 111, 111>> = <<102, 111, 120>>
left:  <<102, 111, 111>>
       -----------------
right: "fox"

In the tests for the ExUnit diff-module there is just one test for binaries which is called “not supported”

  test "not supported" do
    refute_diff(
      <<147, 1, 2, 31>> = <<193, 1, 31>>,
      "-<<147, 1, 2, 31>>-",
      "+<<193, 1, 31>>+"
    )
  end
5

I believe that there is room for exunit’s binary diffing to be improved.

I think this is one case that could clearly be improved because it makes it difficult to compare the left and right side.

If you want to look at prior art, then these series of PR’s/commits that implemented pattern diffing (released in Elixir 1.10) are useful to look at:

I don’t believe there is that level of configurability (and it seems unlikely to be added).

Instead (similar to hauleth) I’d point you to creating functions to construct the error message for assert. And yes you can get them as beautiful as the ExUnit diffs as long as you’re willing to put in the time to implement the appropriate functionality, ExUnit is not privileged in what it can output.

If you do create a binary-focused assertion it might be a good fit for @devonestes assertions library:

3 Likes
6

This seems like a good opportunity to dive into macros, I’ll give it a try as soon as I have the time.

1 Like