I would try using tprof — OTP 29.0.2 (tools 4.2.1) (in memory mode) in setup blocks or find a way to run the test scripts under mix profile.tprof — Mix v1.18.1, note you would need to make sure that all processes are accounted for, not just the test process.

Like

$  MIX_ENV=test mix profile.tprof -e "Mix.Tasks.Test.run([])" --type memory

but for each test file individually.

You can try reducing the max number of parallel tests, Elixir by default uses System.schedulers_online() * 2 - source.
In the LiveView project I work on we had frequent timeouts with this default, so we added an alias to change it to System.schedulers_online() * 1.5, it solved the timeout issues and the total test time didn’t change. It should also reduce the memory usage.

test: "test --warnings-as-errors --max-cases #{round(System.schedulers_online() * 1.5)}",

Yes, this is definitely a valid concern. But sooner or later we’d need to find what makes our memory bloat, and sometimes it pays off to start early.