This thread is just to chat about ways we can improve the Hexdocs ↔ LLM integrations to make it best in class.
I experienced this yesterday:
I was trying to improve my image background removal code that used @kip’s wonderful Image library. My version was using erode and had jagged edges, so using Google’s new Gemini 2.5 I was able to slowly ask it about:
What problem I had initially.
Here are some functions the Image library has.
What do you want to see docs for?
Repeat #2.
After a few back and forths it gave me a great solution that worked better that my original function.
It got me thinking: is there a way to better give these ever-improving AI models the docs and be more helpful to us Elixir developers? In the case of Image, I’m not really an image manipulation guy. My company does not specialize in image manipulation. I just need to remove a chroma key background as reliably as I can as fast as I can. I would have benefitted from a chunky document that listed all the module’s functions, instead of having to copy and paste it.
Is there a more native integration like having mix docs generate a ai text version of the docs that all one beefy chunk we can place into an LLMs context?
How do you guys work with hexdocs and AI? I’m curious if there’s a much better way I’m not aware of. Thanks!
I’m very curious about the actual solution you came up with (tangential to your topic I know). Always looking for a way to improve image. And background removal is something I’ve been investigating.
Honestly I always have the problem of finding the correct documentation, especially when it comes to bigger things such a phoenix.
I don’t think that specialized prompts for a bloated LLM is the way to go, as there is no information where the data was scraped from, this is especially a problem since a library usually has multiple versions, including major versions that change or introduce new things.
I think that fine-tuning a model such as BERT with elixir documentation might be something with a lot of potential, especially since we can make it aware of different versions of libraries. I was investigating this some time ago, but my poor desktop that was built on budget is not the best fit for training models, so iterations are way too slow for experimentation.
And there is Finally, a Replacement for BERT: Introducing ModernBERT which claims better performance on code. So maybe it would be possible to embed both docs and each function source code and then use those for lookups!
It’s definitely possible, as there are already fine-tuned models that understand code pretty well.
I think it’s all about the way you decide to fine-tune and measure the efficiency, for me that honestly is still black magic as I’ve never delved into theory on this topic.
It’s interesting to consider whether RAG would actually beat full-text search (tool use) here. When I’m searching for docs my queries are almost always keyword searches, (library names, function names) - there’s not a whole lot of need for semantic lookups.
Again I really think the general solution will win here - you can do RAG on all of the docs in the world and still lose to a model that finds a blog post or forum thread which solves a given problem.
Interesting, I honestly think fine-tuning models is not the way to go, but rather create a system to provide the correct context to whatever LLM is the best at a given moment/in a given situation.
To that end, it should be ‘somewhat’ trivial to create a solution that always surgically inserts the newest docs of a given library when needed. Be it an MCP server that just fetches it, or a more sophisticated system that knows about the relevant libraries of a project, caches that info, and then retrieves the relevant sub-pages’ info when deemed relevant.
I think it depends on what you plan to be your end result.
I never cared about models guiding me on steps how to write code, but instead I want a flexible way to search for documentation. Either way, elixir core team said they are working on making such a tool happen, we will see what their end result will look like and the approach they decided to take.
In this case I think such a system makes even more sense, as it is essentially using the LLM to fetch the correct docs based on a natural language query instead of relying on potentially outdated training data (you’d have to continuously re-train the model; making it forget past training isn’t really a thing, and it still is likely to mangle some output).
From a tinkerers perspective I’m all for fine-tuning a model to see where it leads us, I definitely see the allure!
And yes, very curious about what the community builds around the core teams’ advancements!
