I’m personally pretty wary of such rules. I think certain types of tests would be worse off by following such rule. For example, in a controller test we might want to test different aspects of a response, what’s the status, what are the headers, what’s the body, etc. To some extent we could write them using one assertion to follow the rule but I think it would be way less readable. In high-level integration tests, especially when hitting the UI, it’d be even less practical to have just one assertion per test. For very low-level tests it seems like a pretty good rule of thumb though.

I’m not a huge fan of strictly following this approach - (or the Rspec one-liner syntax, TBH). Taken to extremes, it can mean slowing down a test suite by a factor of 2-3x or more due to repeated setup.

For concreteness, let’s look at some “off-the-shelf” tests as created by phx.gen.auth:

It’s not complicated to rewrite this to one-per-test:

test "logs the <%= schema.singular %> out and redirects to /", %{conn: conn, <%= schema.singular %>: <%= schema.singular %>} do
  conn = conn |> log_in_<%= schema.singular %>(<%= schema.singular %>) |> delete(Routes.<%= schema.route_helper %>_session_path(conn, :delete))
  assert redirected_to(conn) == "/"
end

test "logs the <%= schema.singular %> out and removes the token", %{conn: conn, <%= schema.singular %>: <%= schema.singular %>} do
  conn = conn |> log_in_<%= schema.singular %>(<%= schema.singular %>) |> delete(Routes.<%= schema.route_helper %>_session_path(conn, :delete))
  refute get_session(conn, :<%= schema.singular %>_token)
end

test "logs the <%= schema.singular %> out and shows a flash message", %{conn: conn, <%= schema.singular %>: <%= schema.singular %>} do
  conn = conn |> log_in_<%= schema.singular %>(<%= schema.singular %>) |> delete(Routes.<%= schema.route_helper %>_session_path(conn, :delete))
  assert get_flash(conn, :info) =~ "Logged out successfully"
end

This makes the test in question take 3x as long to run (because nearly all the work is done 3x) and IMO makes naming the blocks harder.

An approach I’ve seen to that naming problem in Rspec-land is nested describe blocks - which comes with its own “which setup will run” guessing-game, so much so that ExUnit has explicitly made describe non-nestable.

There’s also a philosophical question: what is “one assertion”? For instance, this looks like one assertion:

assert redirected_to(conn) == "/"

but redirected_to can raise, causing the test to fail, if things are not as expected:

• if the conn is in the wrong state
• if the status is not 302
• if the location header is not set

So there are FOUR ways that “single assertion” can cause the test to fail.

One place I can see this technique being useful is when practicing “strong TDD” - the strict “no code can be written without a failing test” approach. Adding “One Assertion Per Test” to that practice means there’s never a point where the coder has to decide “add an assertion to an existing test or add a new test?”

A strict rule there would be useful to overcome analysis paralysis and start typing code.

BUT

That practice also includes a “refactor” step, and IMO the tests are fair game for refactoring. If strong TDD produced the three split-out SessionController tests above, a good refactor would be to note they have identical setup and actions.

I used to do that in Java-land. The idea was a 1:1 mapping of reasons to fail and tests that could fail. In the Java ecosystem, in those days (pre-Rails), true unit tests didn’t even go out of process to hit the db, so there was less of a performance concern with breaking down tests that atomically.

I don’t do that any more because it’s not worth that much “design” in test suites as well as the performance cost. I now look to test suites mostly for their regression value, so that when I’m adding some new feature or refactoring something that there isn’t some surprise dependency I wasn’t aware of that I just broke.