Quick Survey -- Are you doing property testing and if so, what library are you using?

bibekp · March 29, 2018, 10:05pm

I just figured I’d ask the community what they are using for Property Testing these days.

A couple of years ago I used the eqc_ex package which is quviq’s quickcheck for Elixir.

I used this to test out this module

brpandey/hangman/blob/master/lib/hangman/pattern.ex

defmodule Hangman.Pattern do
  @moduledoc """
  Module handles `Hangman` pattern updates given guessed `letter`.

  Given an initial starting state of `-------` and given a letter `a`, 
  it will update the pattern to `a---a--`, for the secret 'avocado'. 

  After letters `o`, and `d`, we are two letters shy of `avocado`.

  ## Example
      iex> p = Pattern.update "-------", "avocado",  "a"
      "a---a--"

      iex> p = Pattern.update p, "avocado",  "o"        
      "a-o-a-o"

      iex> p = Pattern.update p, "avocado",  "d"
      "a-o-ado"

  """

This file has been truncated. show original

with this test file

github.com

brpandey/hangman/blob/master/test/hangman/pattern_eqc.exs

defmodule Hangman.Pattern.Test do
  use ExUnit.Case
  use EQC.ExUnit

  property "the union of a hangman pattern with its inverse pattern equals the original secret" do
    # Property

    # The union of the guessed hangman pattern along with the inverse guessed hangman pattern 
    # equals the original hangman secret pattern

    # We are using inverse functions to design the property

    # Create a generator for the hangman secret lengths
    # ranging from a min secret length of 3 letters to a max of 28
    forall len <- choose(3, 28) do
      # Create a generator that produces a variable sized alphabet char list
      # we use this as the hangman secret char list
      let list <- vector(len, oneof(:lists.seq(?A, ?Z))) do
        secret = to_string(list)

This file has been truncated. show original

I recently just found out recently that Elixir has its own native property testing library, StreamData.

Clearly there are other libraries out there

So my question is: 1) are you doing property testing and if so, 2) what library are you using? And optionally impressions… – Thanks!

gon782 · March 30, 2018, 5:24am

I’m currently working on inserting StreamData into parts of our testing and using it to generate example data for dev work. The reason I chose StreamData was for it’s possible inclusion in Elixir and because I had to choose one to try out. If it felt too cumbersome I’d just use another one.

OvermindDL1 · March 30, 2018, 3:38pm

I prefer the propEr overall. I’m not a fan of quickcheck’s license (at all). StreamData I’ve used recently even though it is not as powerful as proper because I’ve been expecting it to be added to Elixir itself (it really needs to be fleshed out a bit more first though).

benwilson512 · April 3, 2018, 8:18pm

Recently did a bunch of testing with StreamData, worked great. Fantastic experience.

axelson · April 3, 2018, 9:55pm

Another +1 for stream_data here. Haven’t used it very much so far but it’s been working great.

Joseph · April 5, 2018, 1:32pm

I’ve used Quixir, QC for Elixir, PropCheck and StreamData over the past year or so. I’ve gone for StreamData recently because it’s up for inclusion with Elixir (I hope it’ll be easier to do and more common in the future). I’m happy with the API.

keathley · April 8, 2018, 4:04pm

I’ve used StreamData and proper. Recently I’ve been using Proper more because it’s more powerful then StreamData in StreamData’s current state. IMO the true power of property testing lies in model checking which is well supported in Proper. There’s discussions about adding a model checking api to StreamData though. I think that would be a huge win for the community.

axelson · April 8, 2018, 10:25pm

That sounds intriguing, do you have more information about it? I’m having trouble finding out more about this model checking from Proper’s homepage.

Actually I just found the issue you raised on the stream_data github, following along with the conversation, it’s very interesting. Looking forward to playing with some sort of model testing in the future.

cmkarlsson · April 8, 2018, 11:59pm

If you are not too afraid of erlang there is a great book by Fred Hebert about property testing.

http://propertesting.com/

It uses proper and gives a nice introduction from simple property testing to stateful properties.

keathley · April 9, 2018, 1:01am

Fred Hebert provides a fantastic overview of stateful tests and propers StateM module in his property testing book: http://propertesting.com/book_stateful_properties.html