Defining custom generators with PropCheck

Questions / Help
8

Some answers to your questions:

  • aggregate and the like take a value of type test as first parameter. This can something simple such as a boolean expression or something wrapped by PropEr - which is what aggregate and the like do. PropEr calls the variations sometimes outer_test, cooked_test and inner_test.
  • measure provides statistics (such as mean, min and max) suitable for number values (e.g. lengths of lists) whereas the other statistics work on arbitrary values and provide histograms (i.e. counts of matched categories).
  • when_fail wraps the test, it first parameter, and returns it. Otherwise the pipeline wouldn’t work.
  • Elixir binaries are sequences of byte values. Strings in Elixir are those binaries, which follow the utf8 encoding (this is different to Erlang where Strings are lists of (utf8) characters). So, man y binaries are utf8 strings, but not all.
  • nil values do not fit to binaries, since they are not lists. Shrinking of binaries goes towards the empty binary, i.e. <<>>. If you require nil and binaries, the oneof(nil, binary) is a good way to go.

Shrinking or modeling of structs is another topic. Currently, maps are not supported in PropEr natively (see here: https://github.com/manopapad/proper/releases/tag/v1.2 where PropEr 2.0 version is vaguely announced) and I did not implemented my own generator. A few thoughts for discussion on this:

  • you can take a look into the tuple construction which is quite similar (remember, records are tuples of a special form and maps are a nicer version of records). So, at least construction and shrinking strategies can be copied.

  • Considering the advice of jlouis and others (https://medium.com/@jlouis666/quickcheck-advice-c357efb4e7e6#.wdaup7m8b), it may be enough to use a much simpler model (look into the safetyvalve example to get the idea). But this certainly depends on your API and if you need a struct here, this might be a problem.

  • you might emulate a struct by using a tuple (or a single value) in the forall part and then use a simple function or expression to create the struct:

      forall {f <- utf8, l <- utf8} do 
      p = %Person{firstname: f, lastname: l}
      is_valid_person(p)
  end

Take a look into the tree example to see more complex generators with tuples using let_shrink to define growth and shrinking strategies. But my general impression is that you either test rather simply-typed basic functions or you need to test a system with state. In the former case, I doubt that shrinking a person to something with shorter names is a useful strategy to find bugs in your program. In the latter case you need to find a proper model of that system state (which is always simpler than the original one) for testing interesting and relevant properties of the system. Take a look into finite state machines in this case.