Turn module docs and specs into a json definition?

Most of this information is also used in ExDoc templates (used to generate docs like on hexdocs.pm) so that tool’s source could be a good place to start:

https://github.com/elixir-lang/ex_doc/blob/v0.29.1/lib/ex_doc/language.ex

Read up on ExDoc a bit and here is my hacky attempt to get a quick win:

module = Module.concat(["MathOps"])
config = ExDoc.Config.build("Elixir", "1", [source_beam: "beam_dir"])
ExDoc.Retriever.docs_from_modules([module], config)

It returns an empty list however

Any ideas how to get a module info dump (hopefully with attribute details) from ExDoc?

Did some more reading. ExDoc is using Code.fetch_docs under the hood it seems inside defp docs_chunk. Cool I give it a try:

Code.ensure_loaded?(MathOps)
# true
Code.fetch_docs(MathOps)
# {:error, :module_not_found}

Bummer. Well after more reading I find this: Module documentation not immediatelly available after ensuring then module is compiled - #3 by josevalim. So I go down another rabbit hole trying to ensure all modules are done compiling etc get raw beam file etc and it feels overly complicated now. Hopefully someone can provide insight.

I wrote docout to accomplish something very similar. Not sure I understand the issues you’ve run into so far, but maybe it (or the source) could help.

That looks great! If I can’t find a vanilla Elixir way I’ll give it a go.

In regards to Code.fetch_docs I also found this snippet on the writing documentation page:

Code.fetch_docs/1

Elixir stores documentation inside pre-defined chunks in the bytecode. It can be accessed from Elixir by using the Code.fetch_docs/1 function. This also means documentation is only accessed when required and not when modules are loaded by the Virtual Machine. The only downside is that modules defined in-memory, like the ones defined in IEx, cannot have their documentation accessed as they do not have their bytecode written to disk.

So I have to find a way to write the module to beam vm disk, then pass the beam file to Code.fetch_docs/1.

Still trying to reverse engineer Exdoc to see how it does this automagically

Any module from dependencies or your lib folder will have the Docs chunk. You can try any Elixir module to get started Code.fetch_docs(String).

From where / how do I run that though? Previous attempts give:

Code.fetch_docs(MathOps)
# {:error, :module_not_found}

I tried fetch_docs from livebook/local iex/ load + compile module first then fetch etc. I guess I need ELI5

When are you trying to call fetch_docs ? You may need to ensure that compilation of your app source has finished. In docout that is accomplished here: docout/lib/mix/tasks/compile/docout.ex at 5a0ebd0a1cbb77bc9d82b6efd2d0d97d24ebb960 · tfwright/docout · GitHub

Here is local iex:

iex(3)> import Code
Code
iex(4)> Code.compile_file("./MathOps.ex")
[
  {MathOps,
   <<70, 79, 82, 49, 0, 0, 7, 112, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0,
     197, 0, 0, 0, 21, 14, 69, 108, 105, 120, 105, 114, 46, 77, 97, 116, 104,
     79, 112, 115, 8, 95, 95, 105, 110, 102, 111, 95, ...>>}
]
iex(5)> MathOps
MathOps
iex(6)> MathOps.add(1,2)
%{"result" => "3"}
iex(7)> 
nil
iex(8)> Code.ensure_loaded?(MathOps)
true
iex(9)> Code.fetch_docs(MathOps)
{:error, :module_not_found}
iex(10)> 

One thing I have not tried yet is Mix.Task.Compile

Geewiz I finally figured it out and as expected it was something simple. I was starting my local iex session by calling iex directly without any params.

I started iex as iex -S mix and it worked perfectly.

And now fetch_docs actually works hooray!

iex(3)> MathOps.add(1,2)
%{"result" => "3"}
iex(4)> Code.fetch_docs(MathOps)
{:docs_v1, 2, :elixir, "text/markdown",
 %{"en" => "This module provides math operations\n"}, %{},
 [
   {{:function, :add, 2}, 6, ["add(a, b)"],
    %{"en" => "This add function returns the input sum\n\n"}, %{}},
   {{:function, :minus, 2}, 15, ["minus(a, b)"], :none, %{}}
 ]}

And for bonus points I got Exdoc working as well (which provides module specs):

config = ExDoc.Config.build("Elixir", "1", [source_beam: "beam_dir"])
docs = ExDoc.Retriever.docs_from_modules([MathOps], config)

Now I have the info I need to build the json specification