igalic

igalic

How to create a custom StreamData Generator

Hi folks,

I’m trying to transform our tests for AuthToken (a wrapper around JOSE) to ExUnitProperties. But I’m kinda failing at the most basic task of transforming a simple function into a generator. What I’ve done is this:

  defp stream_authtoken_keys(_seed, _range) do
    Stream.repeatedly(&AuthToken.generate_key/0)
  end

  defp gen_authtoken_key() do
    %StreamData{generator: &stream_authtoken_keys/2}
  end

However, when trying to use it…

  describe "keys" do
    property "generate_key/0 returns a valid AES128 key" do
      check all authtoken_key <- gen_authtoken_key()  do
       {:ok, key} = authtoken_key
        assert byte_size(key) == 16
      end
    end
  end

I get the following error:

  1) property keys generate_key/0 returns a valid AES128 key (AuthTokenTest)
     test/authtoken_test.exs:23
     ** (MatchError) no match of right hand side value: #Function<51.91433161/2 in Stream.repeatedly/1>
     code: check all authtoken_key <- gen_authtoken_key()  do
     stacktrace:
       (stream_data) lib/stream_data.ex:203: StreamData.bind_filter/5
       (stream_data) lib/stream_data.ex:346: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data) lib/stream_data.ex:203: StreamData.check_all/6
       test/authtoken_test.exs:24: (test)

Marked As Solved

igalic

igalic

After posting this in elixir (on irc & discord) I got a very decent reply that works out pretty fine, so I’m gonna close this topic for now with it!

First off, trying to generate %StreamData{} myself, is kinda bad form, since it’s basically a private struct…
We looked for alternatives, and found StreamData.constant/0. However, on its own, it always generates the same value.

Which makes a certain amount of sense when you think about how it’s called:

iex(1)> x = StreamData.constant(AuthToken.generate_key) # this is now evaluated, we're done here
%StreamData{generator: #Function<16.32994346/2 in StreamData.constant/1>}
iex(2)> x |> Enum.take(2)                              
[
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>,
  ok: <<109, 179, 255, 84, 16, 160, 239, 132, 198, 157, 218, 254, 13, 160, 196, 137>>
]
iex(3)>

The clue came from @michalmuskala:

You usually construct the opaque type one way or the other, you rarely construct it directly. For example the MapSet struct is opaque, but you can generate it by doing something like map(list(), &MapSet.new/1)

and the code we now have from @ericmj

  defp gen_authtoken_key() do
    StreamData.map(StreamData.list_of(StreamData.constant(:unused_tick)), fn _ ->
      AuthToken.generate_key()
    end)
  end

what we’re doing here is using StreamData.constant/1 as a sort of clock generator, that drives the call of AuthToken.generate_key/0… and it’s nice start, but we’re needlessly generating a list_of which we then map
So let’s update this with all responses below!

  defp gen_authtoken_key() do
    StreamData.unshrinkable(
      StreamData.bind(StreamData.constant(:unused), fn _ ->
        StreamData.constant(AuthToken.generate_key())
      end)
    )
  end

This is much cleaner! Since we use bind/2, which is essentially made for this… better see what’s going on, we can use pipes to extract the flow:

  defp gen_authtoken_key() do
    gen_ticker = fn _ ->
      StreamData.constant(AuthToken.generate_key())
    end

    StreamData.constant(:unused)
    |> StreamData.bind(gen_ticker)
    |> StreamData.unshrinkable()
  end

We feed a stream of :unused into gen_ticker lambda, just so it will be constantly called. Since our call to AuthToken.generate_key/0 is wrapped in said lambda, it will be executed on every call, rather than on ge_ticker’s declaration. Finally, we pass the so generated %StreamData{} into unshrinkable/0 (because it is) before returning it.

Also Liked

whatyouhide

whatyouhide

Elixir Core Team

Seeing this now after Add StreamData.repeatedly to pass in a function as your generator by aforward · Pull Request #104 · whatyouhide/stream_data · GitHub has been opened. Just wanted to mention here too that I also think that a generic way to convert streams to generators would be nice but there is a big limitation: generators are stateless while streams aren’t. Streams are enumerated in order and there is no way for generators to know at which index in the stream it’s at.

@ericmj also your solution can be even simpler as bind + constant is just map:

def repeatedly(fun) do
  StreamData.map(StreamData.constant(nil), fn _ -> fun.() end)
end
ericmj

ericmj

Elixir Core Team

I found a (maybe) cleaner way of doing it that doesn’t involve generating a list:

defp gen_authtoken_key() do
  StreamData.bind(StreamData.constant(:unused), fn _ ->
    StreamData.constant(AuthToken.generate_key())
  end)
end

I haven’t tested this so I can’t verify that it works.

EDIT: @whatyouhide has a more concise solution below: How to create a custom StreamData Generator - #12 by whatyouhide

LostKobrakai

LostKobrakai

It might also be good to make the generator unshrinkable, because shrinking doesn’t seem to make sense in that context.

Where Next?

Popular in Questions Top

RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

Other popular topics Top

hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 55125 245
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

We're in Beta

About us Mission Statement