ExUnit wish - retrospective `:test` env logger level

Chat / Discussions
,
1

In :test env I have the usual logger config :point_down:

# Print only warnings and errors during test
config :logger, level: :warning

which means I have a nice clean console if everything passes (and I use capture_log and with_log when necessary).

Something I would very much enjoy is if I could somehow get the :info logs for failed tests only.

It has to be possible, right? Maybe some kind of custom logger that holds the :info logs in memory and only prints them if current test fails?

If it is possible, it would be really nice to be able to write logger config like this:

My wish :point_down:

config :logger, level: [passed: :warning, failed: :info]
2

Here’s how I have my config/test.exs set up:

config :logger, level: System.get_env("LOG_LEVEL", "warning") |> String.to_existing_atom()

Using this, you could run mix test to get the standard warning/error logs, then you could run LOG_LEVEL=info mix test --failed to get more detailed logs for your failed tests (could also use --trace to make it easier to tell which logs apply to which tests, perhaps).

It’s not quite what you’re asking for, but could get you something close to the desired results with very little work involved.

3

I usually put a Logger.configure(level: level) in the test in question.

2 Likes
4

Would that need to be cleared/reset at the end of the test?

5

These are useful and good techniques.

However, my wish is for a 0-effort solution that will also catch, say, a “flakey” test that fails sometimes and only when run as part of an entire async test suite. I do not want to have to manually change the log level and re-run tests. I want…magic!

1 Like
6

You could generally enable logs and use ExUnit.Case — ExUnit v1.18.3 to show them only in case of errors.

There is however no magic after the fact getting access to already dropped logs.

1 Like
7

In my case I have:

# config/test.exs

config :logger, :console,
  level: String.to_atom(System.get_env("LOGGER_LEVEL", "none"))

# test/test_helper.exs

logs =
  case System.get_env("TEST_LOGS", "all") do
    level when level in ~w[all true] ->
      true

    level when level in ~w[emergency alert critical error warning notice info debug] ->
      [level: String.to_existing_atom(level)]

    "warn" ->
      [level: :warning]

    level when level in ~w[none disabled false] ->
      false
  end

ExUnit.start(
  capture_log: logs
)

And I have 2 separate variable for logs in the terminal and in the error reports.

1 Like
8

Only marginally on topic here but this is one of the beautiful things about tests which are completely deterministic. When designed correctly you can retroactively re-enable (sometimes expensive) logging without changing the result. Obviously this is difficult to implement in practice, and not always possible :slight_smile: