Uniform Elixir Documentation Experience - ExDoc as a Server

I recently came across HexSearch and yesterday found yet another interesting thread on improving the experience of HexDocs (Introduce search across all of HexDocs · Issue #1811 · elixir-lang/ex_doc · GitHub). The latter is apparently aiming at changes at the ExDoc level, requiring documentation for a package to be regenerated to benefit from improvements.

Elixir documentation is generally top notch, specially for the most used libraries, however one of the things that I noticed with time is that the ExDoc/HexDocs experience can be quite inconsistent depending on the version of ExDoc used to build docs for a package.

That means clicking links that take you to another package may change the UI drastically, both aesthetically and functionally. Navigating to older versions of documentation for a given page has a similar effect.

There are advantages to having the documentation built once and never changing for a given release, but there are advantages to having a single “ExDoc server” that can render documentation for arbitrary versions of arbitrary packages, too.

Has this server model been explored in public earlier? I haven’t found any references yet.

I am not really familiar with how ExDoc works and how easy/compatible would it be for a recent version of ExDoc to render docs for arbitrarily old packages – but does that sound plausible and worth investigating further?

In broad lines, I’m thinking a server that would be able to take a Hex package (or perhaps even a Git repo), generate documentation, cache it and serve that. Any updates to the server functionality and design would apply to all pages consistently.

1 Like

ExDoc works primarily by extracting documentation chunks from compiled BEAM code. That means that you should be able to take any BEAM project that compiles, and the same version of :ex_doc, and generate more modern documentation from it.

The additional complexity I foresee is that mix projects can and do customize their generated documentation extensively via configuration in their mix.exs or their rebar3.config files, so you will need to make sure your approach honors them.

This could be as simple as a script that checks out each project, adds your desired version of :ex_doc/unlocks :ex_doc and related dependencies and resets them to your desired version, and tries to do normal docgen!

I think so!

Interesting! Thanks for the insight and information :heart_decoration:

Can ExDoc run from “outside” of a project, that is, without messing with the package dependencies?

If I had just the compiled BEAM code and an isolated environment, say my own temporary Mix project with only ex_doc and related dependencies, could that be used to generate documentation?

Not without messing with dependencies, exactly—the compiled BEAM code by necessity includes the compiled dependencies, if you have access to a completely compiled BEAM project by definition you’ve got the deps in there.

As far as ex_doc supporting being pointed to a folder with precompiled BEAM deps, I don’t know if it has an API for that. I would be surprised, as it’s optimized for the library owner’s developer experience, but you may be able to muck around with internals!

Even if it does support that, I would suggest the full-build solution anyhow; remember that ex_doc does not JUST look at the compiled code, but also the project’s mix/rebar configuration for extra pages, module groups, etc; to build the resulting site. Mix/Rebar build metadata like that don’t end up in the compiled release, so you don’t have enough artifacts to re-generate a full ex_doc site for a library just from precompiled files alone.

1 Like