Semantic diffing is now in main. If anyone is interested in trying it out, please add :mneme as a git dependency and enable semantic diffs in your test.exs configuration:
# mix.exs
defp deps do
[
{:mneme, github: "zachallaun/mneme", only: :test}
]
end
# config/test.exs
config :mneme, defaults: [diff: :semantic]
If you run into any errors, please let me know by opening an issue on GitHub!
Note: Mneme now requires Elixir v1.14 or later.
defp deps do
[
{:mneme, "~> 0.2.1", only: :test}
]
end
In addition to some quality-of-life changes when things go wrong, the big feature that v0.2 brings is a custom diff engine that produces dramatically more readable diffs when auto-assertions are updated. I’m really excited about this and it’s made Mneme much more pleasant to use, at least for me.
I expect there to be some formatting bugs – please report them on GitHub if you encounter anything weird! If you desire the old behavior, you can configure Mneme to continue using textual diffs as before.
Text
Semantic
Text
Semantic
Text
Semantic
... or something in the same way that inspect does at a certain point.
I’ve released a new version (v0.2.2) that disables an optimization that was causing a poor diff in example 2 above, increasing the likelihood that subtrees would become “misaligned”.
v0.2.1
v0.2.2
The new diff does a much better job of reflecting the actual change: the nil is being replaced with a new subtree, and the second subtree is being modified.
I’ve released v0.2.3 which includes fixes for a few bugs.
I’d also like to mention a proposal I just posted on GitHub for the introduction of mix mneme to run tests with Mneme prompts, which would be favored over the current behavior that hijacks mix test. If you have any thoughts/feelings about this, I’d invite you to read the proposal and react/respond on GitHub. 
defp deps do
[
{:mneme, "~> 0.2.6", only: :test}
]
end
In addition to a number of bug fixes, this release updated (and hopefully improved) the formatting for semantic diffs, including the addition of side-by-side diffs if terminal width allows. (This can be disabled with a config option if you prefer stacked diffs.)
An option to “skip” a changed assertion has also been added. This will pass the test, but fail the test run at the end. This can be useful if a test contains multiple auto-assertions, as rejecting the first one would immediately fail that test and prevent the later ones from running.
New demo:
I’m hoping to cut a full release soon, but would like to give folks a few days to try things out and report any bugs that I missed, so I’ve pushed this release as an RC.
defp deps do
[
{:mneme, "~> 0.3.0-rc.0", only: :test}
]
end
auto_assert will now use <- when comparing against falsy values instead of ==, and support for == has been removed.
# old
auto_assert Function.identity(nil) == nil
# new
auto_assert nil <- Function.identity(nil)
When I started on Mneme, I wanted the behavior of auto_assert to exactly match that of assert, including the expectation of a truthy value. Consider the following ExUnit assertions:
# this will fail
assert false = Function.identity(false)
# this would succeed
assert Function.identity(false) == false
# this might succeed or fail
assert some_predicate?()
assert has to expect a truthy value to support that third case, but that case doesn’t make sense for auto_assert. We’re always trying to construct a match pattern, and we use the presence of <- in an AST to disambiguate between a new assertion and one that is being (potentially) updated.
All in all, supporting == comparisons after auto_assert complicated the code and didn’t (in my opinion) add significant value, so I’ve decided to deviate from ExUnit in this particular way and to allow matches on falsy values to succeed.
In addition to a number of fixes, this release is largely about introducing auto_assert_raise, auto_assert_receive, and auto_assert_received.
auto_assert_raisetest "raises when passed invalid arguments" do
auto_assert_raise fn -> some_call!(:invalid) end
end
Running the test gives you two auto-generated options, one with and one without a message:
test "raises when passed invalid arguments" do
auto_assert_raise ArgumentError, fn -> some_call!(:invalid) end
# or
auto_assert_raise ArgumentError, "expected message", fn -> some_call!(:invalid) end
end
auto_assert_receive / auto_assert_receivedExample test case:
test "current process should receive an ack" do
ref = send_request!()
auto_assert_receive()
end
Running the test will prompt you with options for each message received by the current process during the timeout window (default 100ms, same as ExUnit’s assert_receive):
test "current process should receive an ack" do
ref = send_request!()
auto_assert_receive {:ack, ^ref}
end
As with ExUnit, auto_assert_received is the same as auto_assert_receive with a timeout of 0.
As always, any feedback is greatly appreciated!
Yes, that was my original plan, but I decided against it because I feel that Mneme doesn’t provide value when you’re simply asserting against a boolean expression. In that case, you should just be using assert or refute directly.
I also think that it would be more confusing to see your auto_assert being converted to an auto_refute because it began returning a falsy value, instead of auto_assert false <- whatever(). Unifying on auto_assert keeps the return value obvious and explicit.
This is the biggest release yet, including new auto-assertions (mentioned above), quality-of-life improvements, and numerous bug fixes. A couple of additional things made it into the release since the first RC:
:force_update option is now available, forcing patterns to regenerate even when they would succeed. This is especially useful when you’re working on something using maps or structs, since patterns would not re-generate when a key is added. (Thanks to @mindreframer for suggesting this feature!)Please see the changelog linked above for the full list of changes!
The last couple of releases have been smaller fixes and improvements to multi-line string handling, but this release of v0.3.3 has a pretty significant quality-of-life change that I thought it was worth mentioning here that much improves the default pattern selection for maps and structs.
Consider the following new auto-assertion:
user = %{first_name: "Zach", last_name: "Allaun"}
auto_assert put_role(user, :moderator)
When this runs, it will generate two patterns that you can select from:
auto_assert %{} <- put_role(user, :moderator)
auto_assert %{first_name: "Zach", last_name: "Allaun", role: :moderator} <-
put_role(user, :moderator)
Now let’s say you select the second one, but you modify the assertion to only assert on the role:
auto_assert %{role: :moderator} <- put_role(user, :moderator)
Previously, if the result of put_role changed (say, it starts returning roles in a list), Mneme would only generate the “empty” and “full” patterns again, and you’d have to re-edit it down. As of v0.3.3, Mneme will also generate a pattern for the subset of keys you’re asserting on, and select it by default:
This fixes one of my personal pet-peeves. One of the most important tenants guiding Mneme is to suggest the assertions you’d write yourself, and this particular case came up surprisingly often in my own usage.
