Mneme - Snapshot testing tool for the busy programmer

v0.5.0 is now available

In addition to a handful of bug fixes, the main change this release is that Mneme will no longer generate empty maps as suggested patterns; in practice, these were not useful and often led to an explosion of suggestions, especially with nested maps. For instance:

auto_assert %{foo: %{bar: :baz}}

# would previously suggest all of:
auto_assert %{} <- %{foo: %{bar: :baz}}
auto_assert %{foo: %{}} <- %{foo: %{bar: :baz}}
auto_assert %{foo: %{bar: :baz} <- %{foo: %{bar: :baz}}

# but will now only suggest:
auto_assert %{foo: %{bar: :baz} <- %{foo: %{bar: :baz}}
4 Likes

v0.6.0 is now available

This release primarily includes some small fixes to play nicely with the Lexical language server.

Additionally, the restart behavior enabled by using Mneme.start(restart: true) is now always there. @mindreframer, one less thing for you to remember. :smile:

Finally, if you’re using VS Code, I’ve rewritten the VS Code Setup guide to be more supportive of running tests regardless if which language server you’re using.

5 Likes

@zachallaun OMG! :slight_smile: Thanks! Just installing Mneme and using it. You rock!

Rigged up an incredibly quick spec cycle, alongside neovim.io, nushell.sh , and zellij.dev -

map <Leader>sa :wall \| !zellij run -f -- nu -e "nd mix test; zellij action toggle-floating-panes; zellij action write 10; exit";<CR>

(or is this called a “guess-and-check” cycle?)

This command could be used as-is in other forks of vim.
To use alongside bash, replace each ; using &&.

I use a test module as @zachallaun recommends:

defmodule SampleTest do
  use ExUnit.Case, async: true
  use Mneme, action: :accept, default_pattern: :last
  test "abc..." do
    auto_assert [:abc, 123]
  end
end

In this loop I press <space>sa (check your vim’s leader key) -
this opens a floating pane in Zellij which is hidden once the specs end successfully.
The zellij action write 10 causes neovim to reload the file once the spec pane is hidden.

3 Likes

v0.7.0 is now available

This is the first release targeting Elixir 1.17.0 and Erlang/OTP 27. If you are using either of those, please let me know if you encounter anything odd!

Also, some improvements from v0.6.1

While I didn’t make a post for it, v0.6.1 was released a week ago and includes some nice quality-of-life improvements when matching maps. For instance, if you previously had an assertion that looked like this:

auto_assert %{foo: 1, bar: _} <- Map.put(%{bar: 2}, :foo, 1}

and the value of :foo changed, Mneme would suggest a pattern that also asserted the value of :bar:

auto_assert %{foo: 2, bar: 2} <- Map.put(%{bar: 2}, :foo, 2}

As of v0.6.1, Mneme will see that your pattern used _ and will suggest a similar pattern that only updates :foo:

auto_assert %{foo: 2, bar: _} <- Map.put(%{bar: 2}, :foo, 2}

My goal with Mneme is to generate the assertions you would write. When you manually edit a pattern (for instance, by using _ for a map value), Mneme should respect that. If you find other instances where Mneme could have generated a pattern closer to what you wanted, please file an issue!

Preview: File-based snapshots and “matchers”

If you’re interested in what may be coming up next, there are two issues that may be of interest:

These issues include details for something I’m calling matchers (other naming suggestions welcome), which are ways of modifying the behavior of assertions. For instance:

# assert that a larger string contains some substring
auto_assert substring("an error occurred") <-
              capture_log(fn -> Logger.error("some error") end)

# save the generated pattern in `test_snapshots/my_snapshot.exs`
auto_assert snapshot(:my_snapshot) <- my_fun()

# save each result for a list of examples in its own snapshot file
for {name, input} <- examples do
  auto_assert snapshot(name) <- my_fun(input)
end

Any input or feedback on this is greatly appreciated.

2 Likes

v0.8.0 is now available

Thanks to @mudasobwa, the first issue I opened in Mneme has been closed: variables bound in auto_assert patterns are now accessible outside of the expression! This was the main limitation of Mneme vs. plain ExUnit assertions, so I’m very thankful that it’s been resolved.

Prior to this release, you had to use the return value of auto_assert to access anything you were testing. Now, you can bind the result or portions of it inside the assertion pattern:

# previously...
result = auto_assert %{x: 1} <- some_function()

# now...
auto_assert %{x: 1} = result <- some_function()
#                     ^^^^^^
#                     this would previously be a warning or error

auto_assert %{x: x} <- some_function()
#                ^
#                destructuring works as expected
4 Likes

… and now v0.8.1 is also available, fixing a diff bug that’s been eluding me for a while now.

:slightly_smiling_face:

2 Likes

v0.8.2 is now available

This release includes a couple of bug fixes and one small CLI feature.

  • There were some regressions in the last few releases relating to map patterns that could cause many duplicate patterns to be suggested and, in some cases, incorrect patterns that wouldn’t match your actual value. I believe these are now fixed, but please let me know if you run into any weird suggestions!

  • Two new key bindings have been added to the CLI: J and K to jump to the first or last pattern. This can be helpful especially when you have a large, nested data structure, as these can cause Mneme to generate quite a few patterns at various levels of complexity.

2 Likes