Think of it less as “the tests I put in @doc” and more like “it tests that the examples I have in my docs run properly”. The goal is to help you catch changes to your code that invalidate your doc examples, not replace your unit tests with doc tests.

I believe there is.

I’ve mentioned elsewhere that I split my tests into three kinds of testing: unit, integration, and doc tests; each kind of test exists in testing modules which don’t mix kinds of test. This allows me to do something like: @moduletag :unit. Here’s an example of a doctest module:

defmodule DoctestsTest do
  use AuthenticationTestCase, async: true

  @moduletag :doctest
  @moduletag :capture_log

  doctest MscmpSystAuthn
end

I then use the --only flag with mix test, for example mix test --only doctest. This will run the doctests and exclude the unit and integration. When I run mix test --only integration, I get only the integration tests and not the unit or doctests. I have to imagine that the other tag oriented mix test options also work as expected.

However, I have also found that explicitly tagging the doctests like I am isn’t necessary. Even when I don’t add the @moduletag :doctest module attribute, the tag is still respected when including or excluding the doctests as though I added it. That tagging must be happening behind the scenes (or something to that effect). I would expect that this probably would work for your scenario of mixed unit/doctest files if you tagged everything else individually; a @moduletag might override the doctest identification (or not, I’ve not tried it)… but it gives hint that excluding doctest on the command line without any additional tagging might be possible.

OHHHHHHHHHHHHHHHHH I completely misunderstood! Yes, this makes a lot of sense and I quite like this idea. I thought you were saying you moved the actual documentation to a different module from their functions which is what raised my eyebrow