[Elixir v1.12.0] Handling multi-newline exception messages in doctests?

Hi Forum,

Has anyone writing doctests found a way to update their examples that produce ArgumentErrors to capture the new multiline format produced in Elixir 1.12.0? I had some trouble updating an error expectation today because of the double newline.

Here’s what succeeded in Elixir 1.11.4:

@doc ~S"""
  Returns `map` with its keys as atoms, if those atoms already exist.

  Raises `ArgumentError` otherwise.

  ## Examples

      iex> atomize_keys!(%{"oh" => "ooh", "noo" => "noooo"})
      ** (ArgumentError) argument error
"""

But updating to Elixir 1.12.0's reported actual results in this test failure:

Doctest failed: wrong message for ArgumentError
expected:
  "errors were found at the given arguments:\\n\\n  * 1st argument: invalid UTF8 encoding\\n"
actual:
  "errors were found at the given arguments:\n\n  * 1st argument: invalid UTF8 encoding\n"

And if I update the doc tag to interpret escaped characters (i.e., @doc """), the test just fails in the opposite direction:

Doctest failed: wrong message for ArgumentError
expected:
  "errors were found at the given arguments:"
actual:
  "errors were found at the given arguments:\n\n  * 1st argument: invalid UTF8 encoding\n"

…And, yes, escaping the escape fails to improve the situation:

Doctest failed: wrong message for ArgumentError
expected:
  "errors were found at the given arguments:\\n\\n  * 1st argument: invalid UTF8 encoding\\n"
actual:
  "errors were found at the given arguments:\n\n  * 1st argument: invalid UTF8 encoding\n"

Me @ this point: tyra was rooting for escaped backslash

I implemented a new exception as a workaround for my use case, but it feels like a circumstance that does deserve a more generally flexible fix. I mean, what if a library developer does want to use an example of a multiline exception with double newlines in one of their doctests? Clearly, there are cases now where at least ArgumentErrors produce such exceptions, and those cases shouldn’t fail in the test runs. At least, it doesn’t feel like they should ._.

2 Likes

Doctests do not support multi-line exceptions because it is hard for it to know when the exception is over and when you have a new paragraph (as we typically avoid relying on indentation). However, we definitely need to come up with a mechanism to do so… or at least for a subtext match. Please open up an issue!

5 Likes

workaround for me was to use assert_raise without any output check, not ideal to use asserts in doctests but does the job: iex> assert_raise ArgumentError, fn -> atomize_keys!(%{"oh" => "ooh", "noo" => "noooo"}) end