I recently started a new job where I’m working working on a large Phoenix app for the first time. Running tests, or even one test, is very slow. I discovered that this was, in part, due to a test alias that was re-creating the test database on each invocation of $ mix test. Then I realized: phx_new aliases the test task out of the box!
# mix.exs
defp aliases do
# ...
# Generated by phx_new:
test: ["ecto.create --quiet", "ecto.migrate --quiet", "test"]
# ...
end
My assumption is that this is for convenience when running async tests as each new test db will need to be setup and you could change the number of partitions. But, this begs the question: once a project grows very large, how are you supposed to sanely do TDD?
Of course I have ideas—I could rename that alias to something else, like test.all (or something better than that), add a test.prepare to set up a single test db for running non-async tests, and now the test task just runs tests and that would work. But I’m really curious to hear from others (if you’re out there!) who have faced this as well and how you dealt with it.
Thanks!
I’ve created a mix test.prepare that does exactly that (although I called it test.setup). It may have dropped the test database as well to ensure that you get a clean slate (which is very helpful when you’re tweaking a migration file)
Edit: There’s also GitHub - lpil/mix-test.watch: 🎠 Because TDD is awesome that you might be interested in
2 Likes
Great, thanks, good to know I’m on the right track. And yes, definitely test.setup should drop the db! I’m also realizing that that the test alias in the codebase I’m working on has been altered more than I thought—I actually blindly copied the phx_gen version without realizing that it is merely creating and migrating the db whereas in my codebase, it’s reseting the db every time, so that makes a little more sense why phoenix includes this as a default.
mix-test.watch looks nice but I’m personally not a fan of test watchers. I obsessively hit save and have very simple shortcuts for saving and running either the whole file or the test under the cursor. I also switch between TDD, spiking, and even good ol’ “test after” as I go, depending on the scenario.
Thanks for the reply!
@sodapopcan Would be curious how your test speed improves after changing the test alias.
In case the performance remains slow, I’ve found mix test --slowest to be helpful.
--slowest - prints timing information for the N slowest tests. Automatically sets --trace and --preload-modules
— mix test — Mix v1.16.0
Of course I’ve found my biggest testing bottlenecks usually relate to side effects, like interaction with the DB or filesystem. Optimizing these often improves performance significantly.
This really came out of me not noticing that the test alias had actually been altered from what is auto-generated by phx_new. It was doing a complete db reset (drop, create, migrate) which, of course, made running one test insanely slow. I’m in the middle of fixing that now.
We actually have a sync item about mix test --slowest which I will be looking at soon! As for general slow-tests reasons, I’m ok with stuff like general DB interactions slowing things down. I’m not looking for picosecond feedback here, just anywhere under 2 seconds to run a single “slower” unit test would be nice… instead of 40 
1 Like
