Hi all,
I am working on my property based testing library propcheck
. My latest feature is to re-run all failing properties (read: special test cases with their own macro definition) from the last test. So I write the failing property names to a file on disk and control the testing procedure in the next run depending on the file’s content. This works for now, but the reporting via ExUnit
does not consider the skipped successful properties.
To achieve the regular ExUnit
reporting, I generated a tag for each property in my macro with @tag failing_prop: true/false
. The value of the tag depends on the last test run of this particular property. Log output shows that all tags of test module are evaluated before the tests are run, so this part works. But I never succeeded to filter out all properties which were successful at the last run (or vice versa to include only those which failed last time).
My questions are:
- Is it generally possible to generate tags within a macro which sets value of the tag at runtime (= compile team of the
.exs
file)? - Is there any other possibility to inform
ExUnit
about skipped test cases (e.g. via an exception or a return value? - Is my macro construction regarding the
tag
simply wrong?
Here is the relevant snippet of the property
macro:
quote bind_quoted: [name: name, block: block, var: var, opts: opts] do
ExUnit.plural_rule("property", "properties")
prop_name = ExUnit.Case.register_test(__ENV__, :property, name, [])
%{module: module} = __ENV__
if tag_property({module, prop_name, []}) do
@tag failing_prop: true
end
def unquote(prop_name)(unquote(var)) do
p = unquote(block)
mfa = {unquote(module), unquote(prop_name), []}
execute_property(p, mfa, unquote(opts))
:ok
end
end
Thanks,
Klaus.