I’ve verified that the problem is the --no-start flag to mix test. If I run the tests without it, the logger is configured correctly to the :error severity level.
Apparently Application.ensure_all_started(:fun_with_flags) alone is not enough to compensate for the --no-start flag.
