Mneme - Snapshot testing tool for the busy programmer

[zachallaun]

v0.5.0 is now available

• Changelog
• Docs

In addition to a handful of bug fixes, the main change this release is that Mneme will no longer generate empty maps as suggested patterns; in practice, these were not useful and often led to an explosion of suggestions, especially with nested maps. For instance:

auto_assert %{foo: %{bar: :baz}}

# would previously suggest all of:
auto_assert %{} <- %{foo: %{bar: :baz}}
auto_assert %{foo: %{}} <- %{foo: %{bar: :baz}}
auto_assert %{foo: %{bar: :baz} <- %{foo: %{bar: :baz}}

# but will now only suggest:
auto_assert %{foo: %{bar: :baz} <- %{foo: %{bar: :baz}}

[zachallaun]

v0.6.0 is now available

• Changelog
• Docs

This release primarily includes some small fixes to play nicely with the Lexical language server.

Additionally, the restart behavior enabled by using Mneme.start(restart: true) is now always there. @mindreframer, one less thing for you to remember.

Finally, if you’re using VS Code, I’ve rewritten the VS Code Setup guide to be more supportive of running tests regardless if which language server you’re using.

[mindreframer]

@zachallaun OMG! Thanks! Just installing Mneme and using it. You rock!

[c4lliope]

Rigged up an incredibly quick spec cycle, alongside neovim.io, nushell.sh , and zellij.dev -

map <Leader>sa :wall \| !zellij run -f -- nu -e "nd mix test; zellij action toggle-floating-panes; zellij action write 10; exit";<CR>

(or is this called a “guess-and-check” cycle?)

This command could be used as-is in other forks of vim.
To use alongside bash, replace each ; using &&.

I use a test module as @zachallaun recommends:

defmodule SampleTest do
  use ExUnit.Case, async: true
  use Mneme, action: :accept, default_pattern: :last
  test "abc..." do
    auto_assert [:abc, 123]
  end
end

In this loop I press <space>sa (check your vim’s leader key) -
this opens a floating pane in Zellij which is hidden once the specs end successfully.
The zellij action write 10 causes neovim to reload the file once the spec pane is hidden.

[zachallaun]

v0.7.0 is now available

• Changelog
• Docs

This is the first release targeting Elixir 1.17.0 and Erlang/OTP 27. If you are using either of those, please let me know if you encounter anything odd!

Also, some improvements from v0.6.1

While I didn’t make a post for it, v0.6.1 was released a week ago and includes some nice quality-of-life improvements when matching maps. For instance, if you previously had an assertion that looked like this:

auto_assert %{foo: 1, bar: _} <- Map.put(%{bar: 2}, :foo, 1}

and the value of :foo changed, Mneme would suggest a pattern that also asserted the value of :bar:

auto_assert %{foo: 2, bar: 2} <- Map.put(%{bar: 2}, :foo, 2}

As of v0.6.1, Mneme will see that your pattern used _ and will suggest a similar pattern that only updates :foo:

auto_assert %{foo: 2, bar: _} <- Map.put(%{bar: 2}, :foo, 2}

My goal with Mneme is to generate the assertions you would write. When you manually edit a pattern (for instance, by using _ for a map value), Mneme should respect that. If you find other instances where Mneme could have generated a pattern closer to what you wanted, please file an issue!

Preview: File-based snapshots and “matchers”

If you’re interested in what may be coming up next, there are two issues that may be of interest:

• Text-based matching
• File-based snapshots

These issues include details for something I’m calling matchers (other naming suggestions welcome), which are ways of modifying the behavior of assertions. For instance:

# assert that a larger string contains some substring
auto_assert substring("an error occurred") <-
              capture_log(fn -> Logger.error("some error") end)

# save the generated pattern in `test_snapshots/my_snapshot.exs`
auto_assert snapshot(:my_snapshot) <- my_fun()

# save each result for a list of examples in its own snapshot file
for {name, input} <- examples do
  auto_assert snapshot(name) <- my_fun(input)
end

Any input or feedback on this is greatly appreciated.

[zachallaun]

v0.8.0 is now available

• Changelog
• Docs

Thanks to @mudasobwa, the first issue I opened in Mneme has been closed: variables bound in auto_assert patterns are now accessible outside of the expression! This was the main limitation of Mneme vs. plain ExUnit assertions, so I’m very thankful that it’s been resolved.

Prior to this release, you had to use the return value of auto_assert to access anything you were testing. Now, you can bind the result or portions of it inside the assertion pattern:

# previously...
result = auto_assert %{x: 1} <- some_function()

# now...
auto_assert %{x: 1} = result <- some_function()
#                     ^^^^^^
#                     this would previously be a warning or error

auto_assert %{x: x} <- some_function()
#                ^
#                destructuring works as expected

[zachallaun]

… and now v0.8.1 is also available, fixing a diff bug that’s been eluding me for a while now.

[zachallaun]

v0.8.2 is now available

• Changelog
• Docs

This release includes a couple of bug fixes and one small CLI feature.

• There were some regressions in the last few releases relating to map patterns that could cause many duplicate patterns to be suggested and, in some cases, incorrect patterns that wouldn’t match your actual value. I believe these are now fixed, but please let me know if you run into any weird suggestions!

• Two new key bindings have been added to the CLI: J and K to jump to the first or last pattern. This can be helpful especially when you have a large, nested data structure, as these can cause Mneme to generate quite a few patterns at various levels of complexity.

[zachallaun]

v0.9.0-alpha.0 is now available

• Changelog
• Docs

If you like immediate testing feedback, this is for you: mix mneme.watch

$ mix mneme.watch test/my_app/my_test.exs

Inspired by mix test.watch, this is a new testing task that watches your file system for changes, automatically re-running your tests when files are modified. It’s Mneme-aware, which means it can interrupt prompts and save already-accepted tests if a change is detected mid-run.

It’s likely to have some sharp corners, so I’m releasing it as an alpha for now. Please do report any and all bugs or other weirdness that you might come across!

[zachallaun]

v0.9.0 is now available

• Changelog
• Docs

After a few months of dogfooding and smoothing some rough edges, I’m confident enough in mix mneme.watch to ditch the “alpha” moniker in this release.

Give it a try!

[tcoopman]

Lovely, will try it soon.

I must say mneme is really a wonderful package. For people who haven’t tried it yet, I highly recommend it.

[tcoopman]

Just a shoutout: mix mneme.watch is amazing

[zachallaun]

Thank you for saying so! I’ve been wanting to record a new quick demo using it, but have some family in town and so haven’t had time.

I also want to write up some suggestions for its use when I’m able. For instance, I’ve found that I prefer to use it with a single file (mix mneme.watch test/my_lib/some_test.exs) or even a single test (mix mneme.watch test/my_lib/some_test.exs:123) so that it runs really quickly. I plan to add an --exit-on-success flag that would facilitate this even more!

[tcoopman]

What about mix mneme.watch --stale (`mix mneme.watch`: add flag to only run stale tests · Issue #91 · zachallaun/mneme · GitHub)

Isn’t that magical, it already works

[zachallaun]

@tcoopman

Isn’t that magical, it already works

Thanks to the design of Mix and the built-in mix test for being so nicely composable!

[zachallaun]

v0.9.4 is now available

• Changelog
• Docs

This release adds mix mneme.install, an Igniter task for setting up Mneme in a project. Setting up Mneme is now as simple as:

1. Add the dependency: {:mneme, ">= 0.0.0", only: :test}
2. Fetch dependencies: mix deps.get
3. Set up Mneme: MIX_ENV=test mix mneme.install

[zachdaniel]

With that installer defined, if folks are already using igniter, then

MIX_ENV=test mix igniter.install --mneme --only test

should also work, right? If you put only: [:test] in your installer, then just MIX_ENV=test mix igniter.install --mneme would also work. Additionally, mix igniter.install --mneme would yell at the user telling them to rerun in the test env (if you had only: [:test] in your installer).

Not saying you should make that part of your instructions, it’s your library so you do you Just offering it up in case you’re unaware. But I would suggest adding the only: [:test] option to your installer’s info so that it properly yells at users who use mix igniter.install mneme, which is quickly becoming my default way to add packages

[zachallaun]

@zachdaniel

With that installer defined, if folks are already using igniter, then

MIX_ENV=test mix igniter.install --mneme --only test

should also work, right?

I believe it should, yes! At this point, I’m not (yet) assuming that folks will be explicitly using Igniter, so I want the first steps people to see to be the ones you need if you don’t have Igniter installed.

That said, the tip about only: [:test] is great! I didn’t know about that feature and will update the installer to use it.

[zachdaniel]

@zachallaun

At this point, I’m not (yet) assuming that folks will be explicitly using Igniter,

Right, definitely not anywhere near that point yet

[tcoopman]

@zachallaun maybe I’m missing something, but I don’t seem to find in the docs how to run mix test with action: reject. Is the only option to set the CI env variable?