Probably a bit late to the party .

You can now run only the previously failed tests by running mix test --failed
Documentation can be found here

This exact feature was added a few weeks ago. In fact, it was added by Myron Marston (who had implemented the behavior in rspec originally).

Here is the PR for the second half https://github.com/elixir-lang/elixir/pull/7373, it is an awesome demonstration of open source collaboration.

I do this with a custom test formatter (a copy of ExUnit.CLIFormatter) that saves the filenames of the failed tests in the handle_cast function that deals with test failures.

You can get the failed test file and line number from that function by pattern matching:

  def handle_cast({:test_finished, %ExUnit.Test{tags: %{file: file, line: line}, state: {:failed, failures}} = test}, config) do

I add those to the config, and then in print_suite I read them out of the config and save them to a file.

I then have a mix task called retest that runs mix test with the contents of that file as arguments. Note that when passing multiple test files to mix test, you cannot include line numbers (at least in Elixir 1.5).

Then, when I have failures, I just run retest over and over until I’ve fixed all the failures.