Getting closer to the next sizable update to Mneme and wanted to share! After noticing that it can sometimes be difficult to visually parse what changed when an auto-generated value updates, I started working on semantic diffing, using the amazing Difftastic as reference and inspiration.
This has proved more of a rabbit hole than I initially thought, but is still relatively easy thanks to Elixir’s general awesomeness. Today I reached the point where my semantic diff development branch can start using semantic diffs to work on semantic diffs!
For instance, here’s what an update might look like using the latest release:
Here’s the same update using semantic diffs:
There’s still a ways to go before this lands, but I’m hoping that being able to use semantic diffs in development will help expose edge-cases/etc. before I ship the feature.
7 Likes
Semantic diffing is now in main. If anyone is interested in trying it out, please add :mneme as a git dependency and enable semantic diffs in your test.exs configuration:
# mix.exs
defp deps do
[
{:mneme, github: "zachallaun/mneme", only: :test}
]
end
# config/test.exs
config :mneme, defaults: [diff: :semantic]
If you run into any errors, please let me know by opening an issue on GitHub!
2 Likes
v0.2 released
Note: Mneme now requires Elixir v1.14 or later.
defp deps do
[
{:mneme, "~> 0.2.1", only: :test}
]
end
Semantic diffs
In addition to some quality-of-life changes when things go wrong, the big feature that v0.2 brings is a custom diff engine that produces dramatically more readable diffs when auto-assertions are updated. I’m really excited about this and it’s made Mneme much more pleasant to use, at least for me.
I expect there to be some formatting bugs – please report them on GitHub if you encounter anything weird! If you desire the old behavior, you can configure Mneme to continue using textual diffs as before.
Example 1. New assertions highlight only the new value
Text
Semantic
Example 2. Changed assertions highlight deleted/added nodes
Text
Semantic
Example 3. Slightly changed strings call out the changed portion
Text
Semantic
Future work
- Diff quality: These diffs could still be improved. For instance, example 2 highlighted a suboptimal set of square brackets.
- Performance: For large expressions, semantic diffs will take too long and it will fall back to text diffs. I’m sure there’s still some low-hanging fruit for improving performance, however.
- Formatting: I’d like to add side-by-side formatting to decrease the vertical real-estate needed. I’d also like to explore replacing very large, unchanged nodes with
... or something in the same way that inspect does at a certain point.
4 Likes
I’ve released a new version (v0.2.2) that disables an optimization that was causing a poor diff in example 2 above, increasing the likelihood that subtrees would become “misaligned”.
v0.2.1
v0.2.2
The new diff does a much better job of reflecting the actual change: the nil is being replaced with a new subtree, and the second subtree is being modified.
5 Likes
I’ve released v0.2.3 which includes fixes for a few bugs.
I’d also like to mention a proposal I just posted on GitHub for the introduction of mix mneme to run tests with Mneme prompts, which would be favored over the current behavior that hijacks mix test. If you have any thoughts/feelings about this, I’d invite you to read the proposal and react/respond on GitHub. 
v0.2.6 released
defp deps do
[
{:mneme, "~> 0.2.6", only: :test}
]
end
Changes
In addition to a number of bug fixes, this release updated (and hopefully improved) the formatting for semantic diffs, including the addition of side-by-side diffs if terminal width allows. (This can be disabled with a config option if you prefer stacked diffs.)
An option to “skip” a changed assertion has also been added. This will pass the test, but fail the test run at the end. This can be useful if a test contains multiple auto-assertions, as rejecting the first one would immediately fail that test and prevent the later ones from running.
New demo:
2 Likes
