Broader "assert" and "refute" support?

Personally, I think the most important value for test cases is simplicity, so I’m definitely skeptical about support for something like this.

FWIW, about the first example specifically I would argue that you should not be writing assertions against test inputs since they are supposed to be known. Granted I’m not familiar with the DSL shown so maybe I’m misunderstanding something but generally I would only expect the second part of that assertion to be a meaningful test since the former would be testing the test itself.

This is just property-based testing. You declare what type of inputs your functions shouldn’t choke on, and the unittester fuzzes your code for you, then attempts to make minimum viable repros of any failing examples.

  property "union basic correctness" do
    check all multiset1 <- t(term()), multiset2 <- t(term()) do
      result = One9.Multiset.union(multiset1, multiset2)

      check all value <- term() do
        assert One9.Multiset.count_element(result, value) ===
          max(
            One9.Multiset.count_element(multiset1, value),
            One9.Multiset.count_element(multiset2, value)
          )
      end
    end
  end

I had a feeling that was what I was looking at. Nothing against property-based testing as another tool in the arsenal, although personally I haven’t found need for it as of yet in the types of programs I have tested. But I wouldn’t think ExUnit needs to include special support for it.

But also even property tests, as far as I understand, should be able to control inputs at least in terms of types. If one type of input should raise an error, but another should fail gracefully, you need to be able to write a test that isolates both cases, otherwise the distinction is ipso facto irrelevant and shouldn’t be subject to any test. In your example, if the concern is specifically with “well formed” inputs, then I would argue should be a constraint on what is tested in that example, rather than an assertion on the result.

This seems like a reasonable conversation to start on the core mailing list, if you can’t find a prior discussion.

I think property-based testing and other test design philosophy is orthogonal to this discussion:

• Elixir has special forms and operators
• ExUnit tries to provide pretty formatting when recognizing some assertions on the language’s forms
• Whether or not it can do better by recognizing some boolean operator combinations and formatting them specially is a conversation worth having
• Regardless of testing practices

That is correct; they do. The issue arises with situations like those given in my OP.

I’m not entirely sure what you’re getting at, here.

Of course, I can do that (and I do do that, in other tests). But when “A implies B” is exactly what I mean to assert, stuffing some of the conditions for A into the generator headers achieves terseness, at the cost of self-documentingness, which is a very raw deal.

I agree almost entirely—except that I suspect “A implies B” is very unlikely to come up as a natural, relevant, correct thing to test outside of property-based testing (which, I suspect, is what @tfwright has been trying to build an argument for this whole time.)

Agree on the first (property-based) if there is another use case for this syntax, but not so much on the latter (design philosophy). To shift focus back to the latter, my opinion is precisely that support for this would encourage test design I would consider suboptimal on the principles of simplicity and atomicity. I just think tests should generally avoid testing more than one thing at once, regardless of whether that “other thing” is considered an input or not (although if it’s an input then I think it’s particularly bad).

Ironically, the tooling that’s already in there works great for the strictly more complicated case, “A implies B, and not-A implies not-B”—since === works for booleans.

I file B as left, A as right, and I get an extremely educational error message when things break down that colors A green and B red if they ever disagree:

  property "well_formed basic correctness" do
    assert One9.Ms.well_formed?(%{})
    assert One9.Ms.well_formed?(%{42 => 1})
    refute One9.Ms.well_formed?(%{43 => 0})
    refute One9.Ms.well_formed?(%{42 => 1, 43 => 0})

    check all ms <- t(term(), strict: false) do
      assert One9.Ms.well_formed?(ms) === (0 not in Map.values(ms))
    end
  end

^That’s what got me on the train of thought of pulling out just the first half for assertions.

When guards are failing, boolean logic is highlighted with colours. I don’t see any reason why it shouldn’t be supported in ExUnit as well.

зображення709×330 18.5 KB

As @christhekeele has already suggested, please start a discussion on the core mailing list.