Prove - Write simple tests shorter

Your Libraries & OS Mentoring Libraries
1

Prove is a little and experimental lib that provides the macros prove and batch to use in ExUnit.Case.

A prove is just helpful for elementary tests. Prove generates one test with one assert for every prove.

Example:

defmodule NumTest do
  use ExUnit.Case

  import Prove

  defmodule Num do
    def check(0), do: :zero

    def check(x) when is_integer(x) do
      case rem(x, 2) do
        0 -> :even
        1 -> :odd
      end
    end

    def check(_), do: :error
  end

  describe "check/1" do
    prove Num.check(0) == :zero

    batch "returns :odd or :even" do
      prove Num.check(1) == :odd
      prove Num.check(2) == :even
      prove "for big num", Num.check(2_000) == :even
    end

    batch "returns :error" do
      prove Num.check("1") == :error
      prove Num.check(nil) == :error
    end
  end
end

The example above generates the following tests:

$> mix test test/num_test.exs --trace --seed 0

NumTest [test/num_test.exs]
  * prove check/1 (1) (0.00ms) [L#20]
  * prove check/1 returns :odd or :even (1) (0.00ms) [L#23]
  * prove check/1 returns :odd or :even (2) (0.00ms) [L#24]
  * prove check/1 returns :odd or :even for big num (1) (0.00ms) [L#25]
  * prove check/1 returns :error (1) (0.00ms) [L#29]
  * prove check/1 returns :error (2) (0.00ms) [L#30]


Finished in 0.08 seconds (0.00s async, 0.08s sync)
6 proves, 0 failures

Randomized with seed 0

The benefit of prove is that tests with multiple asserts can be avoided.
The example above with regular tests:

...
  describe "check/1" do
    test "returns :zero" do
      assert Num.check(0) == :zero
    end

    test "returns :odd or :even" do
      assert Num.check(1) == :odd
      assert Num.check(2) == :even
      assert Num.check(2_000) == :even
    end

    test "returns :error" do
      assert Num.check("1") == :error
      assert Num.check(nil) == :error
    end
  end
...

$> mix test test/num_test.exs --trace --seed 0

NumTest [test/num_test.exs]
  * test check/1 returns :zero (0.00ms) [L#36]
  * test check/1 returns :odd or :even (0.00ms) [L#40]
  * test check/1 returns :error (0.00ms) [L#46]


Finished in 0.03 seconds (0.00s async, 0.03s sync)
3 tests, 0 failures

Randomized with seed 0

You can find another example in datix.

The disadvantage of these macros is that the tests are containing fewer descriptions. For this reason and also if a prove looks too complicated, a regular test is to prefer.

6 Likes
2

I wonder how will the feature of breaking apart multiple prove statements to multiple test + assert blocks work with stateful tests, e.g. Phoenix+Ecto app tests? Example:

describe "create and list users" do
  prove Repo.insert(...) == {:ok, %User{}}
  prove Repo.all(User) == [%User{}]
end

:point_up: will this break Ecto’s sandbox? If this actually gets broken down to 2x test + assert blocks then likely the second test/block will fail?

3

Yes this would break Ecto’s sandbox and even when not the proves/tests would be executed in random order. So, prove makes just sense for little and pure functions. If you have to think about how to write a test with prove then test is the better choice.

1 Like
4

When using batch/prove there’s no more value using prove rather than test/assert?

    batch "returns :error" do
      prove Num.check("1") == :error
      prove Num.check(nil) == :error
    end

    test "returns :error" do
      assert Num.check("1") == :error
      assert Num.check(nil) == :error
    end

You don’t save any code here while saving code is the purpose of prove, if I understood correctly. It’s a little weird, at least to have introduced batch. Or maybe I missed something.

3 Likes
5

I think the advantage of prove in these situations is, that it converts each prove to its own test case.

In your example with test you end up with one test case and it has two asserts in it. You need to check on failure which assert failed and the ones after the failed one are not executed / evaluated while the first one fails.

The prove example though creates two test cases, each with one assert. Both test cases are independent and on failure you instantly know which assert / prove is the problem.

And then batch just allows one to group multiple proves so they all start with the same “prefix”, so you get a bit more context.

At least that’s how I understood everything.

2 Likes
6

That would make sense but that is not what @Marcus wrote above. He wrote the equivalent of batch with three proves as one test with three asserts.

1 Like
7

But you’re right, I see that the resulting output for batch/prove is as you described. Thank you.

8

@thojanssens1 it works as @IloSophiep describes. But you’re right @thojanssens1, the documentation doesn’t explain it well and neither does my post. I will update the docs. @thojanssens1, @IloSophiep thanks for your posts.

3 Likes
9

Nice little idea!

1 Like