Explicit vs DRY tests

I always prefer explicit tests. Yes, they are more tedious to write, but much easier to understand when they start to fail.

Also, if trying to keep tests as DRY as production code then at one point they might get so complex that they need their own tests.

Weird how nobody did tests to get best from both worlds:

for n <- [1, 2, 4, 7, 8] do
  n = Macro.escape(n)

  test "fizz_buzz(#{n}) returns the input number as a string" do
    n = unquote(n)
    assert fizz_buzz(n) == "#{n}"
  end
end

Basically have your code generate tests. And if one fails you’ll have an exact test title + assert diff showing exactly what’s wrong.

I do such tests any chance I get because it helps cover several potentially-gotcha values.

But if we’re going into that territory then I’ll side with @hauleth – when you need to prove properties of your code then property testing is what you want. And it’s not an overkill at all, you use it 4-5 times and you get used to it. I have never spent more than 20-30 minutes adding property tests once I learned stream_data (which admittedly took me a weekend of tinkering, though it was well worth it).

You can get around this by making a loop around the “test” instead of the “assert” and putting the test value into the test name, but I generally don’t bother with the loop because of reasons stated above.

For me stream_data is a way to go from day one. It is “must have” just like credo and ex_doc.

After all, I really consider it as a nice feature proposal for ExUnit

True! Maybe because it tests all the case from 1 to 100 (or more easily), which is maybe not the purpose of a readable (as a spec) unit test.

I think it’s a bit too much to have a dependency to stream_data for a trivial unit test, I will avoid to use it for now

I will start to apply a combination of the version 1 and 3

My approach, as your is IMHO too repeating the implementation:

defmodule Kata.FizzBuzzTest do
  use ExUnit.Case
  use ExUnitProperties

  property "multiples of 3 contains `Fizz`" do
    check all n <- integer() do
      result = fizz_buzz(n * 3)
      assert String.contains?(result, "Fizz")
    end
  end

  property "multiples of 5 contains `Buzz`" do
    check all n <- integer() do
      result = fizz_buzz(n * 5)
      assert String.contains?(result, "Buzz")
    end
  end

  property "returned value is a binary" do
    check all n <- integer() do
      assert is_binary(fizz_buzz(n))
    end
  end

  property "result for 'plain' values is stringified integer" do
    check all n <- integer(), rem(n, 3) * rem(n, 5) != 0 do
      assert fizz_buzz(n) == Integer.to_string(n)
    end
  end
end

There probably could be some additional properties like the above

ExUnit outputs me this in an error case:

  1) test &fizz_buzz/1: multiples of 3 and 5 returns Fizz (Kata.FizzBuzz.Test)
     test/kata/fizz_buzz_test.exs:27
     Assertion with == failed
     code:  assert fizz_buzz(n) == "FizzBuzz"
     left:  "FizzBug"
     right: "FizzBuzz"

It point the problem: we would liked to know the value of n in this case

I’m surprised ExUnit does not already print out the values of the variables used in the failing code.

I agree here. It’s nice having very explicit, clear failures and not having to guess which one caused a failure. The assert message may provide a clear failure in this case. I think one should default to minimum abstraction in your tests, if there are no other pressing concerns like complexity of function under test.

I usually look at the failure message from my tests to decide whether I like the style of the test.

I think I am an outlier when it comes to tests, but I would not include any logic to generate assertions in unit tests. I would just choose one type of input for each type of output, with a test for each. If this logic was extra critical or issues keep popping up with unexpected inputs, I guess I would consider adding some sort of property based test to supplement the unit tests, but I would consider carefully before doing it.