Correct, ollama is making the embeddings. Right now the default model is nomic-embed-text.
What does it do in general? Do you use same embedding on each query through the llm tool use and thats how it does queries? So a simple one step embedding query?
There are two exposed tools via this MCP server, search and fetch.
Fetch downloads the hexdocs as if you would be running mix hex.docs ... yourself but then in addition to that, it ports the html docs to markdown, chunks them and creates embeddings from those chunks with specific metadata.
Search then takes your query, either for a specific package or in general and converts that to an embedding which is then compared to existing chunk queries via L2/euclidian distance.
Does that answer your question?
This is all pretty new to me so if you have suggestions on a better way I’m all ears.
There are optimizations I do want to make, like supplying a URL from the embedding meta to the relevant hexdocs so the LLM can decide to reach out to the specific docs and fetch the entire page if it can’t find what it’s looking for.
I was first thinking to statically make embeddings to avoid needing to run ollama, but you still need to embed each query anyways.
The embedding needs to be configureable at least. If you are already paying for claude, you want to be able to use their embedding models over the api for the mcp work for (most likely much higher quality results than small local models). Any such use will be a rounding error compared to the large model token usage.
Be able to implement a behaviour and expose it as a config.
Right yes, agreed it would be nice to be able to swap out embedding backends!
I’d be curious how much better claude/openai/etc. embeddings would be vs. using local embeddings. That would be an interesting experiment.
The one main thing I like about the current implementation though is that everything is offline, minus downloading hex docs. So anyone can use this without worry about sending sensitive queries about their code to an embedding provider. But yes, I think giving that choice to the user makes sense.
I tried running the mcp server through npx, which didn’t work initially for me, so I tested out the binary directly, which gives this error when doing a search:
~/src/hexdocs-mcp/bin (main*) » ./hexdocs_mcp_macos_arm search ash --query "ash changeset" tgk@Torkilds-MacBook-Pro-M4
Searching for "ash changeset" in ash latest...
21:51:48.574 module=application_controller function=info_exited/3 [notice] Application hexdocs_mcp exited: exited in: HexdocsMcp.CLI.start(:normal, [])
** (EXIT) an exception was raised:
** (Exqlite.Error) no such function: vec_f32
Fetching the docs and building the embeddings work just fine with the binary, it’s just the search functionality that appears to not work, it appears that sqlite-vec extension is not injected judging from the error.
Pulling the repo and running mix hex.docs.mcp search ash --query "ash changeset" works just fine, both on HEAD and at the 0.3.0 tag.
I’m on MacOS ARM btw.
Thank you. Can you try 0.2.0 and let me know if you have any issues?
Were you running an older version before or is this your first time setting it up?
No worries, great initiative btw! 
Yeah, this was the first time, I also wiped ~/.hexdocs_mcp when switching between HEAD & 0.3.0 etc, so that wasn’t an issue.
But yes, the binary for 0.2.0 appears to be working correctly 
Okay that’s great to know! I’ll take a look and see if I can reproduce/put out a fix. Of course it works on my machine. 
I just released v0.3.1 that hopefully resolves the issue with sqlite_vec not being downloaded properly with the included release binary. Please give it a try and LMK if you have any issues!
I have yet another release, v0.4.0, which brings some minor speed improvements. The gist is that if you’re fetching a library that you’ve already fetched, but a different version, it could be faster. Not a ton, but a little bit. For example, on my machine, fetching ash went from ~1m50s to about ~1m10s. It’s still not great but it’s an improvement!
In case you’re curious, I do this by hashing each content chunk that’s embedded and then prior to performing an embedding, check if a previous chunk with that same hash exists or not.
Additionally, I’ve found that the windows build is broken. At least I think it is because that’s what the new CI suite shows (link). If anyone has the time to help and has a windows machine, a PR would be much appreciated!
