Thank you so much! I bought it on PragProg.
I’m still struggling to see the counter-examples with Elixir, though. I tried
File.read!("_build/propcheck.ctex")
|> :erlang.binary_to_term()
and got an ArgumentError.
UPDATE
I found that the couter-examples file _build/propcheck.ctex is a DETS table, whose keys are mfargs, but I’m confusing about the values. They seem to always be [[]].
I’m not 100% sure of it but looking at propcheck/lib/counterstrike.ex at bcdb467546c5853ef7d529bd987f2638bb23ddbc · alfert/propcheck · GitHub, it appears that the format is {mfa, counterexamples}. I would expect mfa to be the code behind the failing property, and counterexamples a list of the specific inputs that can make it fail.
So the counterexamples being [[]] means that there is a single counterexample when the input is []. These are passed directly to PropEr when re-running the property: propcheck/lib/properties.ex at 3e81043dd090085f1bfd5bb9f70d44acc5294e1c · alfert/propcheck · GitHub – the library then does the rest.
The way I handled them in the rebar3 plugin is a bit different, but I rely on the same sort of functionality. The PropEr library hands the counterexample back to you upon failure, and if you send the same arguments back you should get the same failure out if nothing changed.
Thanks. It seems my toy property fails when the input is an empty list, and Elixir treats [[]] as an empty iodata so prints an empty line.
Now I’m facing another problem: PropCheck seems not doing shrinking. I’m following the book and implemented the biggest(list) function as
def biggest([head|tail]), do: biggest(tail, head)
defp biggest([], max), do: max
defp biggest([head|tail], max) when head > max, do: biggest(tail, head)
defp biggest([head|tail], max) when head < max, do: biggest(tail, max)
The minimal input that can cause a failure is a list with 2 identical elements, but when I run
PROPCHECK_VERBOSE=1 mix test
PropCheck outputs
Failed: After 24 test(s).
An exception was raised:
** (FunctionClauseError) no function clause matching in Pbt.BiggestTest.biggest/2
Stacktrace:
[6, 6, 1, -3, -5, -3, -3]
Shrinking (0 time(s))
[6, 6, 1, -3, -5, -3, -3]
.................................................................
1) property finds biggest element (Pbt.BiggestTest)
test/biggest_test.exs:5
Property Elixir.Pbt.BiggestTest.property finds biggest element() failed. Counter-Example is:
[[6, 6, 1, -3, -5, -3, -3]]
Counter example stored.
code: nil
stacktrace:
(propcheck 1.2.2) lib/properties.ex:227: PropCheck.Properties.handle_check_results/2
test/biggest_test.exs:5: (test)
Interestingly, when I code the property in Erlang, PropEr is doing shrinking correctly.