How to create a custom StreamData Generator

igalic

Hi folks,

I’m trying to transform our tests for AuthToken (a wrapper around JOSE) to ExUnitProperties. But I’m kinda failing at the most basic task of transforming a simple function into a generator. What I’ve done is this:

  defp stream_authtoken_keys(_seed, _range) do
    Stream.repeatedly(&AuthToken.generate_key/0)
  end

  defp gen_authtoken_key() do
    %StreamData{generator: &stream_authtoken_keys/2}
  end

However, when trying to use it…

  describe "keys" do
    property "generate_key/0 returns a valid AES128 key" do
      check all authtoken_key <- gen_authtoken_key()  do
       {:ok, key} = authtoken_key
        assert byte_size(key) == 16
      end
    end
  end

I get the following error:

  1) property keys generate_key/0 returns a valid AES128 key (AuthTokenTest)
     test/authtoken_test.exs:23
     ** (MatchError) no match of right hand side value: #Function<51.91433161/2 in Stream.repeatedly/1>
     code: check all authtoken_key <- gen_authtoken_key()  do
     stacktrace:
       (stream_data) lib/stream_data.ex:203: StreamData.bind_filter/5
       (stream_data) lib/stream_data.ex:346: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data) lib/stream_data.ex:203: StreamData.check_all/6
       test/authtoken_test.exs:24: (test)

11 comments

#testing #property-testing

0 4259 11

2018-06-04 23:19:47 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

DanCouper

check all authtoken_key <- gen_authtoken_key()  do
  assert byte_size(key) == 16
end

Just as simple as authtoken_key rather than key?

Post #1

igalic

After posting this in elixir (on irc & discord) I got a very decent reply that works out pretty fine, so I’m gonna close this topic for now with it!

First off, trying to generate %StreamData{} myself, is kinda bad form, since it’s basically a private struct…
We looked for alternatives, and found StreamData.constant/0. However, on its own, it always generates the same value.

Which makes a certain amount of sense when you think about how it’s called:

iex(1)> x = StreamData.constant(AuthToken.generate_key) # this is now evaluated, we're done here
%StreamData{generator: #Function<16.32994346/2 in StreamData.constant/1>}
iex(2)> x |> Enum.take(2)                              
[
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>,
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>
]
iex(3)>

The clue came from @michalmuskala:

You usually construct the opaque type one way or the other, you rarely construct it directly. For example the MapSet struct is opaque, but you can generate it by doing something like map(list(), &MapSet.new/1)

and the code we now have from @ericmj

  defp gen_authtoken_key() do
    StreamData.map(StreamData.list_of(StreamData.constant(:unused_tick)), fn _ ->
      AuthToken.generate_key()
    end)
  end

what we’re doing here is using StreamData.constant/1 as a sort of clock generator, that drives the call of AuthToken.generate_key/0… and it’s nice start, but we’re needlessly generating a list_of which we then map
So let’s update this with all responses below!

  defp gen_authtoken_key() do
    StreamData.unshrinkable(
      StreamData.bind(StreamData.constant(:unused), fn _ ->
        StreamData.constant(AuthToken.generate_key())
      end)
    )
  end

This is much cleaner! Since we use bind/2, which is essentially made for this… better see what’s going on, we can use pipes to extract the flow:

  defp gen_authtoken_key() do
    gen_ticker = fn _ ->
      StreamData.constant(AuthToken.generate_key())
    end

    StreamData.constant(:unused)
    |> StreamData.bind(gen_ticker)
    |> StreamData.unshrinkable()
  end

We feed a stream of :unused into gen_ticker lambda, just so it will be constantly called. Since our call to AuthToken.generate_key/0 is wrapped in said lambda, it will be executed on every call, rather than on ge_ticker’s declaration. Finally, we pass the so generated %StreamData{} into unshrinkable/0 (because it is) before returning it.

Post #2

igalic

i think that was just an awkward copy/paste error… and i should fix this in the original post

Post #3

igalic

{:ok, key} = authtoken_key

was missing in my paste (not the error causing code)

Post #4

ericmj

Elixir Core Team

I found a (maybe) cleaner way of doing it that doesn’t involve generating a list:

defp gen_authtoken_key() do
  StreamData.bind(StreamData.constant(:unused), fn _ ->
    StreamData.constant(AuthToken.generate_key())
  end)
end

I haven’t tested this so I can’t verify that it works.

EDIT: @whatyouhide has a more concise solution below: How to create a custom StreamData Generator - #12 by whatyouhide

Post #5

LostKobrakai

It might also be good to make the generator unshrinkable, because shrinking doesn’t seem to make sense in that context.

Post #6

igalic

I really couldn’t figure this out from the documentation…
maybe we should add something to the docs that helps people figure out how to create custom generators

@michalmuskala suggested to have StreamData.repeatedly/1. I was thinking maybe a better way would be to have a way of converting a Stream to a StreamData.

Post #7

LostKobrakai

I found it quite easy to build more complex generators out of the basic generators already in the streamdata module, but I’ve to say that your usecase really doesn’t seem very well supported by the current functions.

Post #8

anthonator

@igalic’s approach only seems to work if you use the generator in a check. If you try to use it in a gen you run into a the issue of the value being considered a constant so it’s the same on each run through. Any way around this?

Post #9