Property-Based Testing with PropEr, Erlang, and Elixir (Pragprog)

Learning Resources Books
, , ,
23

The Philly Elixir Group will be using Property-Based Testing with PropEr, Erlang, and Elixir for their next online book club, starting Jan 28th. If anyone outside Philly would like to join us, there are very welcome!

2 Likes
24

Thank you so much! I bought it on PragProg.

I’m still struggling to see the counter-examples with Elixir, though. I tried

File.read!("_build/propcheck.ctex")
|> :erlang.binary_to_term()

and got an ArgumentError.

UPDATE

I found that the couter-examples file _build/propcheck.ctex is a DETS table, whose keys are mfargs, but I’m confusing about the values. They seem to always be [[]].

25

I’m not 100% sure of it but looking at https://github.com/alfert/propcheck/blob/bcdb467546c5853ef7d529bd987f2638bb23ddbc/lib/counterstrike.ex, it appears that the format is {mfa, counterexamples}. I would expect mfa to be the code behind the failing property, and counterexamples a list of the specific inputs that can make it fail.

So the counterexamples being [[]] means that there is a single counterexample when the input is []. These are passed directly to PropEr when re-running the property: https://github.com/alfert/propcheck/blob/3e81043dd090085f1bfd5bb9f70d44acc5294e1c/lib/properties.ex#L177 – the library then does the rest.

The way I handled them in the rebar3 plugin is a bit different, but I rely on the same sort of functionality. The PropEr library hands the counterexample back to you upon failure, and if you send the same arguments back you should get the same failure out if nothing changed.

1 Like
26

Thanks. It seems my toy property fails when the input is an empty list, and Elixir treats [[]] as an empty iodata so prints an empty line.

Now I’m facing another problem: PropCheck seems not doing shrinking. I’m following the book and implemented the biggest(list) function as

def biggest([head|tail]), do: biggest(tail, head)

defp biggest([], max), do: max
defp biggest([head|tail], max) when head > max, do: biggest(tail, head)
defp biggest([head|tail], max) when head < max, do: biggest(tail, max)

The minimal input that can cause a failure is a list with 2 identical elements, but when I run

PROPCHECK_VERBOSE=1 mix test

PropCheck outputs

Failed: After 24 test(s).
An exception was raised:
** (FunctionClauseError) no function clause matching in Pbt.BiggestTest.biggest/2
Stacktrace:


[6, 6, 1, -3, -5, -3, -3]

Shrinking (0 time(s))
[6, 6, 1, -3, -5, -3, -3]
.................................................................

  1) property finds biggest element (Pbt.BiggestTest)
     test/biggest_test.exs:5
     Property Elixir.Pbt.BiggestTest.property finds biggest element() failed. Counter-Example is:
     [[6, 6, 1, -3, -5, -3, -3]]
     
     Counter example stored.
     
     code: nil
     stacktrace:
       (propcheck 1.2.2) lib/properties.ex:227: PropCheck.Properties.handle_check_results/2
       test/biggest_test.exs:5: (test)

Interestingly, when I code the property in Erlang, PropEr is doing shrinking correctly.