Ash.Generator property testing for invalid inputs

Ash Forum Ash Questions / Help
1

The new how to guide on testing resources has some helpful ideas on how to test that actions handle valid inputs.

What’s the recommended way to test that invalid inputs don’t slip through? There’s no Ash.Generator.invalid_action_input/3.

I could just gen all my own invalid inputs (using the “wrong” generator for each attribute) but I wondered if there was a better way?

2

I think the best approach here would be to modify the existing generators with invalid attribute generators. For example:

    property "doesn't accept maps as input to text" do
      user = Domain.create_user!()

      check all(input <- Ash.Generator.action_input(Tweet, :create, %{text: StreamData.fixed_map(%{a: 1})})) do
        {text, other_inputs} = Map.pop!(input, :text)
        Domain.create_tweet!(text, other_inputs, authorize?: false, actor: user)
      end
    end

This particular one is a bit of a silly example, but the point was to illustrate that individual generators can be overridden with generators that produce whatever you like to test specific conditions.

Does that seem like it fits the bill of what you’re looking for?

1 Like
3

Yep, that works - thanks. That’s what I have been doing (except using constant/1 instead of fixed_map/1).

I guess I was wondering if it’s advantageous to have a generator which streams over all the inputs and can provide bad inputs for each attribute one-at-a-time, then shrinks to just the ones which fail. But that might cause a combinatorial explosion of potential cases which isn’t feasible to test.

Anyway, your solution is good and works fine. Thanks :slight_smile:

4

Its an interesting idea, but yeah I’ve not done anything fancier than try out specific invalid scenarios that I want to test