I recently noticed in our codebase had some iex> usage in doc strings, but those modules did not have a corresponding ExUnit.DocTest.doctest/2. Some of the authors of these helpful iex> examples did not realize there was an extra step to get this examples checked as test (when I was new Elixir, I also did not realize this).
Is there any way to automate checks for this? Or is there something we might add to our Credo configure?
Curious if there might be a way to help avoid missing out on the feedback and checks. I love DocTest examples and want to keep promote their use.
Thanks in advance for thoughts or points.
3 Likes
For now, I created this small documentation PR: Update docs-tests-and-with.markdown by kacorvus · Pull Request #1669 · elixir-lang/elixir-lang.github.com · GitHub . Maybe others can suggest a better improvement to highlight this area.
Can anybody point out an example credo rule that finds all elixir modules containing a specific string?
You can try combining the utilities that doctest uses to extract doc + extract test from doc (elixir/doc_test.ex at v1.14.3 · elixir-lang/elixir · GitHub) to detect presence of a doctest in a module, with the source tree traversal implementation of credo checks like the one which ensures modules have a doc (credo/module_doc.ex at master · rrrene/credo · GitHub), to implement finding all doctests. If that works, that gets you half way there.
The doctest utility seems to work with compiled code (Code — Elixir v1.12.3). The credo check may be working with ast before compilation. I am unsure if you can use the utilities used in doctest with compiled modules at the time credo is running.
If you can use macros to define modules, then my favorite idea is a macro ‘deftestmodulewithdoctest’ which adds doctest call at the top of the test module definition.
