Doctor - keep your documentation healthy

documentation
#1

A problem that I have constantly had with large software projects is ensuring that there is adequate documentation coverage throughout the project so that new team members (and existing team members of course) can jump right in and understand what is going on and what things are supposed to do.

In order to scratch that itch in Elixir, I created Doctor. Doctor will validate that your documentation coverage (this includes module docs, function docs and type specs) is above your desired threshold. Think of unit test coverage minimum…but for documentation. The thresholds are also configurable to suit your team’s needs via a .doctor.exs file in your project.

I currently use this on my Elixir projects as part of CI/CD (Doctor will return a non zero exit code if validation fails) so that documentation is always treated like a first class citizen and it is never a matter of “I’ll go back and add docs later” because let’s face it I probably will forget ;).

Feel free to leave comments or feedback and curious if people have similar itches! This is still a work in progress but a 1.0.0 release should be out by the end of June.

Links to the project can be found here:

And a sample output for documentation coverage of the Phoenix project can be found here:

Doctor file not found. Using defaults.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
DOC COV  SPEC COV  MODULE                                   FILE                                                                  FUNCTIONS  NO DOCS  NO SPECS  MODULE DOC
100%     0%        Mix.Phoenix                              lib/mix/phoenix.ex                                                    18         0        18        YES
0%       0%        Mix.Phoenix.Context                      lib/mix/phoenix/context.ex                                            6          6        6         YES
63%      0%        Mix.Phoenix.Schema                       lib/mix/phoenix/schema.ex                                             8          3        8         YES
100%     0%        Mix.Tasks.Compile.Phoenix                lib/mix/tasks/compile.phoenix.ex                                      2          0        2         YES
100%     0%        Mix.Tasks.Phx.Digest.Clean               lib/mix/tasks/phx.digest.clean.ex                                     1          0        1         YES
100%     0%        Mix.Tasks.Phx.Digest                     lib/mix/tasks/phx.digest.ex                                           1          0        1         YES
100%     0%        Mix.Tasks.Phx                            lib/mix/tasks/phx.ex                                                  1          0        1         YES
100%     0%        Mix.Tasks.Phx.Gen.Cert                   lib/mix/tasks/phx.gen.cert.ex                                         2          0        2         YES
100%     0%        Mix.Tasks.Phx.Gen.Channel                lib/mix/tasks/phx.gen.channel.ex                                      1          0        1         YES
86%      0%        Mix.Tasks.Phx.Gen.Context                lib/mix/tasks/phx.gen.context.ex                                      7          1        7         YES
100%     0%        Mix.Tasks.Phx.Gen.Embedded               lib/mix/tasks/phx.gen.embedded.ex                                     6          0        6         YES
100%     0%        Mix.Tasks.Phx.Gen.Html                   lib/mix/tasks/phx.gen.html.ex                                         4          0        4         YES
100%     0%        Mix.Tasks.Phx.Gen.Json                   lib/mix/tasks/phx.gen.json.ex                                         4          0        4         YES
100%     0%        Mix.Tasks.Phx.Gen.Presence               lib/mix/tasks/phx.gen.presence.ex                                     1          0        1         YES
100%     0%        Mix.Tasks.Phx.Gen.Schema                 lib/mix/tasks/phx.gen.schema.ex                                       7          0        7         YES
100%     0%        Mix.Tasks.Phx.Gen.Secret                 lib/mix/tasks/phx.gen.secret.ex                                       1          0        1         YES
100%     0%        Mix.Tasks.Phx.Routes                     lib/mix/tasks/phx.routes.ex                                           1          0        1         YES
100%     0%        Mix.Tasks.Phx.Server                     lib/mix/tasks/phx.server.ex                                           1          0        1         YES
100%     0%        Phoenix                                  lib/phoenix.ex                                                        3          0        3         YES
100%     0%        Phoenix.Channel                          lib/phoenix/channel.ex                                                12         0        12        YES
100%     0%        Phoenix.Channel.Server                   lib/phoenix/channel/server.ex                                         17         0        17        YES
100%     0%        Phoenix.CodeReloader                     lib/phoenix/code_reloader.ex                                          2          0        2         YES
40%      0%        Phoenix.CodeReloader.Proxy               lib/phoenix/code_reloader/proxy.ex                                    5          3        5         YES
33%      0%        Phoenix.CodeReloader.Server              lib/phoenix/code_reloader/server.ex                                   6          4        6         YES
88%      0%        Phoenix.Config                           lib/phoenix/config.ex                                                 8          1        8         YES
100%     0%        Phoenix.Controller                       lib/phoenix/controller.ex                                             42         0        42        YES
100%     0%        Phoenix.Controller.Pipeline              lib/phoenix/controller/pipeline.ex                                    6          0        6         YES
100%     0%        Phoenix.Digester                         lib/phoenix/digester.ex                                               2          0        2         YES
100%     0%        Phoenix.Endpoint                         lib/phoenix/endpoint.ex                                               25         0        25        YES
100%     0%        Phoenix.Endpoint.Cowboy2Adapter          lib/phoenix/endpoint/cowboy2_adapter.ex                               3          0        3         YES
0%       0%        Phoenix.Endpoint.Cowboy2Handler          lib/phoenix/endpoint/cowboy2_handler.ex                               5          5        5         YES
100%     0%        Phoenix.Endpoint.CowboyAdapter           lib/phoenix/endpoint/cowboy_adapter.ex                                2          0        2         YES
0%       0%        Phoenix.Endpoint.CowboyWebSocket         lib/phoenix/endpoint/cowboy_websocket.ex                              8          8        8         YES
100%     0%        Phoenix.Endpoint.RenderErrors            lib/phoenix/endpoint/render_errors.ex                                 3          0        3         YES
93%      0%        Phoenix.Endpoint.Supervisor              lib/phoenix/endpoint/supervisor.ex                                    15         1        15        YES
0%       0%        Phoenix.Endpoint.Watcher                 lib/phoenix/endpoint/watcher.ex                                       2          2        2         YES
NA       NA        Plug.Exception.Phoenix.ActionClauseErro  lib/phoenix/exceptions.ex                                             0          0        0         NO
NA       NA        Phoenix.NotAcceptableError               lib/phoenix/exceptions.ex                                             0          0        0         YES
100%     0%        Phoenix.MissingParamError                lib/phoenix/exceptions.ex                                             1          0        1         YES
0%       0%        Phoenix.ActionClauseError                lib/phoenix/exceptions.ex                                             2          2        2         NO
60%      0%        Phoenix.Logger                           lib/phoenix/logger.ex                                                 5          2        5         YES
83%      0%        Phoenix.Naming                           lib/phoenix/naming.ex                                                 6          1        6         YES
NA       NA        Phoenix.Param.Map                        lib/phoenix/param.ex                                                  0          0        0         NO
NA       NA        Phoenix.Param.Integer                    lib/phoenix/param.ex                                                  0          0        0         NO
NA       NA        Phoenix.Param.BitString                  lib/phoenix/param.ex                                                  0          0        0         NO
NA       NA        Phoenix.Param.Atom                       lib/phoenix/param.ex                                                  0          0        0         NO
NA       NA        Phoenix.Param.Any                        lib/phoenix/param.ex                                                  0          0        0         NO
0%       0%        Phoenix.Param                            lib/phoenix/param.ex                                                  1          1        1         YES
100%     0%        Phoenix.Presence                         lib/phoenix/presence.ex                                               17         0        17        YES
NA       NA        Phoenix.Router.NoRouteError              lib/phoenix/router.ex                                                 0          0        0         YES
100%     0%        Phoenix.Router                           lib/phoenix/router.ex                                                 11         0        11        YES
100%     0%        Phoenix.Router.ConsoleFormatter          lib/phoenix/router/console_formatter.ex                               1          0        1         YES
95%      0%        Phoenix.Router.Helpers                   lib/phoenix/router/helpers.ex                                         20         1        20        YES
100%     0%        Phoenix.Router.Resource                  lib/phoenix/router/resource.ex                                        1          0        1         YES
100%     0%        Phoenix.Router.Route                     lib/phoenix/router/route.ex                                           5          0        5         YES
100%     0%        Phoenix.Router.Scope                     lib/phoenix/router/scope.ex                                           9          0        9         YES
NA       NA        Phoenix.Socket.InvalidMessageError       lib/phoenix/socket.ex                                                 0          0        0         YES
57%      0%        Phoenix.Socket                           lib/phoenix/socket.ex                                                 14         6        14        YES
NA       NA        Phoenix.Socket.Reply                     lib/phoenix/socket/message.ex                                         0          0        0         YES
100%     0%        Phoenix.Socket.Message                   lib/phoenix/socket/message.ex                                         1          0        1         YES
NA       NA        Phoenix.Socket.Broadcast                 lib/phoenix/socket/message.ex                                         0          0        0         YES
50%      0%        Phoenix.Socket.PoolSupervisor            lib/phoenix/socket/pool_supervisor.ex                                 4          2        4         YES
NA       NA        Phoenix.Socket.Serializer                lib/phoenix/socket/serializer.ex                                      0          0        0         YES
0%       0%        Phoenix.Socket.V1.JSONSerializer         lib/phoenix/socket/serializers/v1_json_serializer.ex                  3          3        3         YES
0%       0%        Phoenix.Socket.V2.JSONSerializer         lib/phoenix/socket/serializers/v2_json_serializer.ex                  3          3        3         YES
100%     0%        Phoenix.Socket.Transport                 lib/phoenix/socket/transport.ex                                       6          0        6         YES
NA       NA        Phoenix.Template.UndefinedError          lib/phoenix/template.ex                                               0          0        0         YES
100%     0%        Phoenix.Template                         lib/phoenix/template.ex                                               11         0        11        YES
0%       0%        Phoenix.Template.EExEngine               lib/phoenix/template/eex_engine.ex                                    1          1        1         YES
NA       NA        Phoenix.Template.Engine                  lib/phoenix/template/engine.ex                                        0          0        0         YES
0%       0%        Phoenix.Template.ExsEngine               lib/phoenix/template/exs_engine.ex                                    1          1        1         YES
NA       NA        Phoenix.ChannelTest.NoopSerializer       lib/phoenix/test/channel_test.ex                                      0          0        0         YES
100%     0%        Phoenix.ChannelTest                      lib/phoenix/test/channel_test.ex                                      19         0        19        YES
100%     0%        Phoenix.ConnTest                         lib/phoenix/test/conn_test.ex                                         17         0        17        YES
100%     0%        Phoenix.Token                            lib/phoenix/token.ex                                                  2          0        2         YES
67%      0%        Phoenix.Transports.LongPoll              lib/phoenix/transports/long_poll.ex                                   3          1        3         YES
50%      0%        Phoenix.Transports.LongPoll.Server       lib/phoenix/transports/long_poll_server.ex                            4          2        4         YES
0%       0%        Phoenix.Transports.WebSocket             lib/phoenix/transports/websocket.ex                                   2          2        2         YES
100%     0%        Phoenix.View                             lib/phoenix/view.ex                                                   9          0        9         YES
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Summary:

Passed Modules: 72
Failed Modules: 7
Total Doc Coverage: 85.1%
Total Spec Coverage: 0.0%

Doctor validation has passed!```
4 Likes
#2

Why create new tool instead of improving inch? What are advantages (except separate count for specs)?

#3

Good question @hauleth! Before creating Doctor I did come across inch and played around with it. At the time (this may have changed since of course) it did not seem like inch took into account module docs and type specs which was something that I personally wanted to take into account. In addition, it did not seem like “doc coverage” was a primary concern of inch. On the other hand for me, having configurable thresholds which I can validate via a CI/CD pipeline was. This way, commit to commit, the coverage of documentation does not drop. It helps ensure that a team upholds their own best practices.

Looking at the inch README again now it doesn’t look like our philosophies overlap a great deal and in fact, I wouldn’t be surprised if teams used both tools as they offer different benefits.