Would you enable ExUnit's `--trace` option in a CI environment (GitHub)? (poll)

Chat / Discussions
, , , ,
1

Coworker and I are at an impasse and would appreciate the community’s vote to resolve it. :slight_smile:

Would you enable --trace option in a CI environment (GitHub)?
  • Yes, --trace option in a CI environment has its uses.
  • No, --trace option in a CI environment makes no sense.
0 voters

Elixir’s test-runner has a --trace option which does the following:

  1. it discards all of Elixir’s concurrency abilities and (slowly) runs everything sequentially, one-by-one
  2. it prints the test-names (here, here, etc.), even the passing ones. (The failed tests are, of course, always printed, even without --trace.)
  3. sets the test-timeout to :infinity. It’s tracing, so this is expected/welcome (in dev/local environment). But not desired in a CI env as it can blow up the bill/costs.

Here’re coworker and my positions:

  • Coworker’s argument for tracing is effectively the 2. above. In his words, it “gives a better visual understanding which tests are running”.
  • My position against it are that:
    • Not only are downsides of 1. and 3. not worth it, but one can also
    • just look at the test file to see which tests are running. Moreover, one shouldn’t even care about the names of the passing tests, abiding by the Rule of Silence.
2 Likes
2

You can use a formatter like junit_formatter to have test information without having to use --trace.

2 Likes
3

Some companies require you use --trace for auditing purposes. I personally think that is stupid but I’ve worked at a place where it was mandated by, I think, the investors? I dunno. Don’t @ me I’m just messenger.

4

Just to be clear, as the docs mention this flag only disables running tests concurrently with each other. Processes are still preempted and so on.

It sounds like the purpose of this is to debug tests which are somehow interfering with each other in a way that they should not be.

5

Yes trace is more for debugging the tests. If some company requires --trace for some reason, I would run both versions in CI, so the CI still runs with tests in concurrency that can fail and detect if something is messing with shared state.

2 Likes