bradley

bradley

Improving AI Tooling with Ash Documentation

I’ve been developing Ash resources and noticed that while AI tools work well when I provide existing examples, they struggle when creating new resources or tackling less-documented use cases. I’m exploring ways to improve AI responses by incorporating the Ash documentation.

One idea is to use the newly released Model Context Protocol from Anthropic, combined with a Retrieval-Augmented Generation (RAG) system, AWS Bedrock Knowledge Bases, or tools like Cline. This would involve downloading HEX docs for the Ash libraries, feeding them into a RAG system or knowledge base, and making them accessible for AI queries.

However, there are some challenges:

  • Ash documentation is extensive, covering multiple libraries within the ecosystem.
  • The docs are constantly updated, so I’d need a way to keep them in sync.

Alternatively, I’m considering whether the Model Context Protocol could directly access Ash documentation without requiring the setup and maintenance of a separate RAG system.

While this problem is most noticeable with Ash, it’s not unique to it—it applies to any HEX library. However, Ash is where I’ve experienced the most issues, as LLMs tend to wildly make things up when attempting to assist with it.

Before diving in, I’d like to ask:

  • Has anyone tried a similar approach?
  • Are there simpler solutions or existing tools for integrating HEX library documentation with LLMs effectively?

Any insights or suggestions would be much appreciated! :folded_hands:

Most Liked

dmitriid

dmitriid

For those interested, Tidewave works:

  • follow the installation instructions
  • if you’re running in wsl, install mcp_proxy (see docs, the python one worked for me) on windows, not in wsl
  • add MCP server to your editor (e.g. Cursor)
     "tidewave-phoenix": {
        "command": "mcp-proxy",
        "args": [
          "http://localhost:4000/tidewave/mcp"
        ]
      }
    

Enjoy.

Here’s a simple rules file I now have in my Cursor that is always loaded:

# Ash Resources and Documentation

- Always use the eval tool to list or introspect Ash resources.
- When documentation about the Ash framework or its features is requested, use package_docs_search to retrieve relevant HexDocs.

And it works:

mjrusso

mjrusso

Try this out: https://hex2txt.fly.dev

You’ll want to use the hex2context notebook, which tries to find and include only the most relevant sections of documentation. (I just shipped this ten minutes ago, and there’s tons of opportunities for improvement. Note that the notebook computes all embeddings locally, but swapping in paid models should be relatively trivial. I haven’t tested different embeddings models out yet; any real-world feedback would be super useful.)

To run the notebook: Run - Livebook.dev

Direct link to source: hex2txt/notebooks/hex2context.livemd at main · mjrusso/hex2txt · GitHub

bradley

bradley

Just wanted to share an update, @mjrusso. I’ve been using hex2text alongside Cursor’s documentation indexing, and it’s been working pretty well. All I do is point it to the hex2text link for the library I need, and voila! I’d really love to see something like this integrated into Hex proper.

Overall, this workflow is decent, but Cursor’s index doesn’t always get utilized when I expect it to, and it still generates a fair amount of incorrect Ash code. That said, it’s still the best solution I’ve tried so far—better than Windsurfer, Copilot, Zed, Cline, and Aider.

Also, cursor just added support for MCP which is a bonus.

Where Next?

Popular in Questions Top

_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48475 226
New
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

We're in Beta

About us Mission Statement