ExUnit wish - retrospective `:test` env logger level

Here’s how I have my config/test.exs set up:

config :logger, level: System.get_env("LOG_LEVEL", "warning") |> String.to_existing_atom()

Using this, you could run mix test to get the standard warning/error logs, then you could run LOG_LEVEL=info mix test --failed to get more detailed logs for your failed tests (could also use --trace to make it easier to tell which logs apply to which tests, perhaps).

It’s not quite what you’re asking for, but could get you something close to the desired results with very little work involved.

I usually put a Logger.configure(level: level) in the test in question.

Would that need to be cleared/reset at the end of the test?

These are useful and good techniques.

However, my wish is for a 0-effort solution that will also catch, say, a “flakey” test that fails sometimes and only when run as part of an entire async test suite. I do not want to have to manually change the log level and re-run tests. I want…magic!

You could generally enable logs and use ExUnit.Case — ExUnit v1.20.2 to show them only in case of errors.

There is however no magic after the fact getting access to already dropped logs.

In my case I have:

# config/test.exs

config :logger, :console,
  level: String.to_atom(System.get_env("LOGGER_LEVEL", "none"))

# test/test_helper.exs

logs =
  case System.get_env("TEST_LOGS", "all") do
    level when level in ~w[all true] ->
      true

    level when level in ~w[emergency alert critical error warning notice info debug] ->
      [level: String.to_existing_atom(level)]

    "warn" ->
      [level: :warning]

    level when level in ~w[none disabled false] ->
      false
  end

ExUnit.start(
  capture_log: logs
)

And I have 2 separate variable for logs in the terminal and in the error reports.

Only marginally on topic here but this is one of the beautiful things about tests which are completely deterministic. When designed correctly you can retroactively re-enable (sometimes expensive) logging without changing the result. Obviously this is difficult to implement in practice, and not always possible