It may have been a rogue code_reloader: true setting in config/test.exs for one of the endpoints that we just found
code_reloader: true
config/test.exs