James_E
I see that the current ExUnit source code has support for rich failure messages on a small whitelist of “recognized” assertion patterns, namely:
assert match?(left, right),assert left = rightrefute match?(left, right),refute left = rightassert left === right,refute left !== right,assert left !== right,refute left === right,assert left == right,refute left != right,assert left != right,refute left == right,assert left >= right,refute left < right,assert left > right,refute left <= rightassert left <= right,refute left > rightassert left < right,refute left >= rightassert left ~= rightrefute left ~= right
Would there be any interest in a pull request adding support for an additional pattern:
refute left and not right(implication, right must be true whenever left is)
This pattern is coming up repeatedly as the most robust, developer-intent-communicating form for some unittests I’m writing:
property "delete returns a well-formed multiset whenever input is well-formed" do
check all ms <- t(term(), strict: false),
value <- term(),
count <- one_of([non_negative_integer(), constant(:all)]) do
result = One9.Ms.delete(ms, value, count)
refute One9.Ms.well_formed?(ms) and not One9.Ms.well_formed?(result)
end
end
property "put 0 copies doesn't corrupt struct" do
check all multiset <- t(term()), value <- term() do
result = One9.Multiset.put(multiset, value, 0)
assert One9.Ms.well_formed?(result.counts)
assert One9.Multiset.equals?(result, multiset)
refute One9.Multiset.member?(result, value) and not One9.Multiset.member?(multiset, value)
end
end
And having these tests augmented with a little bit of extra verbiage and awesome debug formatting, like the comparison checks currently have, would be a minor boon.
Trending in Discussions
The obligatory hello world thread!
Who are you and where are you from? :stuck_out_tongue:
New
Hey there,
It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
Hi everyone :waving_hand:
I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New
Hi everyone,
I have 8 years of production experience with Erlang/OTP and I’m just starting to get to know Elixir. What would you learn f...
New
A remark @Garrison made…
..has already inspired a thread for the technical side of reclaiming our internet, but there is of course anot...
New
Other Trending Topics
Hey there,
I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first.
...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #elixirconf-us
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #elixirconf-eu
- #processes
- #security
- #api
- #forms
- #performance











Showing Posts 1 to 8- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
tfwright
Personally, I think the most important value for test cases is simplicity, so I’m definitely skeptical about support for something like this.
FWIW, about the first example specifically I would argue that you should not be writing assertions against test inputs since they are supposed to be known. Granted I’m not familiar with the DSL shown so maybe I’m misunderstanding something but generally I would only expect the second part of that assertion to be a meaningful test since the former would be testing the test itself.
James_E
This is just property-based testing. You declare what type of inputs your functions shouldn’t choke on, and the unittester fuzzes your code for you, then attempts to make minimum viable repros of any failing examples.
Tangent
I do use “old fashioned hardcoded-input” testing liberally within the doctests, and I have a few exemplar edge or obscure prototype cases hard-coded in other unit tests. But having a per se stream of well-formed spaghetti being thrown against the walls of my code is extremely valuable and I use it for the heavy lifting.
tfwright
I had a feeling that was what I was looking at. Nothing against property-based testing as another tool in the arsenal, although personally I haven’t found need for it as of yet in the types of programs I have tested. But I wouldn’t think ExUnit needs to include special support for it.
But also even property tests, as far as I understand, should be able to control inputs at least in terms of types. If one type of input should raise an error, but another should fail gracefully, you need to be able to write a test that isolates both cases, otherwise the distinction is ipso facto irrelevant and shouldn’t be subject to any test. In your example, if the concern is specifically with “well formed” inputs, then I would argue should be a constraint on what is tested in that example, rather than an assertion on the result.
christhekeele
This seems like a reasonable conversation to start on the core mailing list, if you can’t find a prior discussion.
I think property-based testing and other test design philosophy is orthogonal to this discussion:
James_E
That is correct; they do. The issue arises with situations like those given in my OP.
I’m not entirely sure what you’re getting at, here.
Of course, I can do that (and I do do that, in other tests). But when “A implies B” is exactly what I mean to assert, stuffing some of the conditions for A into the generator headers achieves terseness, at the cost of self-documentingness, which is a very raw deal.
I agree almost entirely—except that I suspect “A implies B” is very unlikely to come up as a natural, relevant, correct thing to test outside of property-based testing (which, I suspect, is what @tfwright has been trying to build an argument for this whole time.)
tfwright
Agree on the first (property-based) if there is another use case for this syntax, but not so much on the latter (design philosophy). To shift focus back to the latter, my opinion is precisely that support for this would encourage test design I would consider suboptimal on the principles of simplicity and atomicity. I just think tests should generally avoid testing more than one thing at once, regardless of whether that “other thing” is considered an input or not (although if it’s an input then I think it’s particularly bad).
James_E
Ironically, the tooling that’s already in there works great for the strictly more complicated case, “A implies B, and not-A implies not-B”—since
===works for booleans.I file B as
left, A asright, and I get an extremely educational error message when things break down that colors A green and B red if they ever disagree:^That’s what got me on the train of thought of pulling out just the first half for assertions.
fuelen
When guards are failing, boolean logic is highlighted with colours. I don’t see any reason why it shouldn’t be supported in ExUnit as well.
As @christhekeele has already suggested, please start a discussion on the core mailing list.