I am having a hard time coming up with negative tests. I believe it can be done with filters, but I haven’t been able to come up good working case.
Say I have this test case:
property “Positive Test” do
check all variable1 <- StreamData.integer(1…5) do
assert Enum.member?([1,2,3,4,5], variable1) == true
end
end
I would like to perform a negative test
property “Negative Test” do
check all variable1 <- !(StreamData.integer(1…5)) do
assert Enum.member?([1,2,3,4,5], variable1) == false
end
end
where !(StreamData.integer(1…5)) is any input that is the “opposite” of the valid input.
I generated a bad negative test that simply generates any term and that test has always passed, but I understand it is not mutually exclusive.
Two thoughts it would be nice if we had two types of “opposite”:
- One that is bound to the same variable type
- The other or any term.
In this way I could show that if a valid input exists it works, if an invalid input of the same data type is provided it fails, and all other results crash.
This would allow for the ability to potentially have a test cases that could catch any case that was expected to crash and didn’t (hardware failure?).
I would really like to take all of the properties from the positive property test and negate it, but I don’t know if this would really work for more complex properties.
All suggestion would be greatly appriciated!
