Generate docs markdown similar to ePub

Hi,
Oficial docs got this wonderful feature to download the ePub version that we can chuck to our kindles and read it. Now we are in the age of AI coding assistants and format like ePub is something LLMs do not like to see as context. What they do like is markdown, xml, json.

I propose in the similar fashion to ePub, to have a markdown version of the full docs of Phoenix and LiveView so the scrapers or manually can automatically and easily download the markdown to use as context for AI Coding tools.
Maybe some type of versioning and update daytime markers would be nice too.

Thanks

2 Likes

I had the same thought and your post pushed me to make a quick proof-of-concept:

3 Likes

I’m thinking a ZIP can be useful when you want to provide the whole context, but being able to copy just a module or even a function’s docs from the web UI could be handy too?

Thanks, I think it’s a great start. Not sure about the zip though. It reduces the size but complicates the actual end goal - getting the context. It adds friction.

Usually, in AI coding tools like Cursor or the terminal based ā€œaiderā€ you just provide the url and the tool scrapes it and the more complex it is to scrape, the lesser quality results you get. Therefore making people think Elixir is not easy and even AI tools can’t help much.

My thinking is if you make an AI ready content ( like markdown version of the HTML doc page) easy to be accessed by these AI coding scrapers, the higher quality will be the LLM model’s reply.
It literally could be just an additional link for a markdown version ( next to the ePub version link)

For now I’m using the conversion with pandoc through terminal after downloading the epub version first.

pandoc -i /Documents/knowledge_base/elixir_phoenix/Phoenix1.7.18.epub -o /Documents/knowledge_base/
elixir_phoenix/Phoenix_1.7.18.md

Related: Can we create an llms.txt file for Phoenix? Similar to Svelte's llms.txt file Ā· Issue #6076 Ā· phoenixframework/phoenix Ā· GitHub

For now, I don’t think Phoenix itself should do anything here. The ExDoc PR was already linked and there’s also https://hex2txt.fly.dev/.