I’m testing some complex ecto sql builder stuff. So I pass data to generate an ecto query and I want to validate that it produces the right query. So, I’m using the Repo.to_sql function to generate SQL and then asserting that matches the expected SQL.
That works OK, but when the SQL is long, it makes it hard to understand which parts of the query differ. I wrote something that can format SQL, because I thought it’d improve the diff output that exunit provides. However, that just added some more whitespace to a single line diff between the two SQL queries. Here’s an example:
What I would like is to have a test failure that would wrap on the newlines instead. Ideally, it would look something like this with the diff coloring elixir does:
1) test formats SQL (SQLFormatterTest)
test/sql_formatter_test.exs:5
Assertion with == failed
code: assert SQLFormatter.format("SELECT * FROM users") == "SELECT\n *\nFROM\n user\nWHERE\n id = 1\n"
left: """
SELECT
*
FROM
users
"""
right: """
SELECT
*
FROM
user
WHERE
id = 1
"""
stacktrace:
test/sql_formatter_test.exs:6: (test)
I’m curious if anyone has any suggestions for how to get better test output?
The problem I run into here is that format_test_failure seems to escape \n as \\n. AFAIK (and I would love to be proved wrong) I think you have to do the formatting yourself.
This is a cobbled together version from a formatter I have. I was hooking into the standard formatter and using format_test_failure so I had to remove that which of course removes all the meta data like test name and line number (though it’s easily added as all of that info is available between error and meta). All this has is left and right but it does work if you want to use it as a starting point!
Again, hopefully I’m wrong and there is an easier way.
defmodule Formatter do
use GenServer
def init(_opts) do
{:ok, %{failures: []}}
end
def handle_cast({:test_finished, %{state: {:failed, errors}} = _test} = _event, state) do
error =
errors
|> Enum.map(fn {:error, error, _meta} ->
left =
String.split(error.left, "\n")
|> Enum.map(& " " <> &1)
|> Enum.join("\n")
right =
String.split(error.right, "\n")
|> Enum.map(&" " <> &1)
|> Enum.join("\n")
"left: \"\"\"\n" <> left <> "\"\"\"\n\nright: \"\"\"\n" <> right <> "\"\"\""
end)
|> Enum.join("\n")
state = %{state | failures: [error | state.failures]}
{:noreply, state}
end
def handle_cast({:suite_finished, _} = _event, state) do
if Enum.any?(state.failures) do
failures =
state.failures
|> Enum.reverse()
|> Enum.join("\n")
IO.puts("""
Failures:
#{failures}
""")
end
{:noreply, state}
end
def handle_cast(_event, state) do
{:noreply, state}
end
end
ExUnit.start(formatters: [Formatter])
defmodule TestIt do
use ExUnit.Case
test "thing" do
assert """
line 1
line 2
line 3
""" == """
line oops
line 2
line 3
"""
end
end
I’ve been doing similar things at work for showing meaningful diffs between 2 JSON files. I had to normalize both: sort keys (json keys don’t have order), pretty print, and only then diff.
Thanks for the suggestions. I was definitely in the headspace of thinking I’d have to write a custom assertion. I forgot ex_unit even had formatters, so that will be a useful area to explore.
