Issue with ExUnit coloring

monkeygroover

I have an issue since upgrading to Elixir 1.15, some of our tests that use capture log are failing because the output contains ansi color escape codes. I added the following before each ExUnit.start() in each app in the umbrella to try to fix this:

ExUnit.configure(
exclude: :skip,
formatters: [ExUnit.CLIFormatter, JUnitFormatter],
capture_log: true,
colors: [enabled: false]
)

however it is now not deterministically fixing it. some runs the tests pass, and some runs the tests fail with the escape codes. Are these settings per umbrella app, or is it possible they share some global state and depending on ordering there is a race somewhere?

6 comments

#exunit

0 664 6

2023-09-11 12:35:53 UTC

Showing Posts 1 to 6

Show Best Posts
Show All (oldest first)
Show All (newest first)

eksperimental

You can use IO.ANSI.format(chardata, false) , convert it to a string and match on it.

iex> IO.ANSI.format(["Hello, ", :red, :bright, "world!"], true) |> to_string()
"Hello, \e[31m\e[1mworld!\e[0m"

iex> IO.ANSI.format(["Hello, ", :red, :bright, "world!"], false)
[[[], "Hello, "], "world!"]

iex> IO.ANSI.format(["Hello, ", :red, :bright, "world!"], false) |> to_string()
"Hello, world!"

Post #1

monkeygroover

Thanks, but that isn’t the problem i’m asking about. i’m not trying to make the tests pass. The issue is that colors: [enabled: false] doesn’t appear to work reliably. some test runs there are no colors and it works fine, other times the colors appear. This seems strange so there may be something odd happening with the recent logger changes.

Post #2

monkeygroover

I can see that in the code these settings are stored using Application.put_env. and shared between ExUnit tests, so i suspect it can be affected like this thread Using Application.get_env / Application.put_env in ExUnit tests

a test i’m seeing fail frequently is an async: true one…

Edit: Although, still seeing the issue even if i make it async false, mysterious.

Post #3

linusdm

Do you get a consistent error if you run the tests with a seed of a test-run that has resulted in a failure before? (using the --seed option to mix test)

Never dealt with this before, but I see that the value of the ExUnit colors config for :enabled is taken from IO.ANSI.enabled?. So I think you can also disables colors by doing such a thing:

config :elixir, ansi_enabled: false

in config/test.exs.

But that probably uglifies all your CLI output too, now that I think about it…

Post #4

michallepicki

Not sure if it’ll help, but I’m on Elixir 1.15.5 and have this in config/test.exs:

config :logger, :default_formatter,
  colors: [
    enabled: false
  ]

Post #5

monkeygroover

Yes, thank you, that helped. I’m not completely sure why i was seeing seemingly intermittent failures, but i’ve gone through and reviewed/updated all the logging settings to use the latest :default_formatter changes and then added using your suggested colors section in test and it’s all working nicely again. thanks.

Post #6

— All posts loaded —