KristerV
Hexdocs search engine for devs
How I currently use Hexdocs
I use hexdocs all day every day, but finding the right module and function takes too long even with my bookmark system.
Too much scrolling, reading, clicking around for the right function. I already know what function I need!
Solution
So I’m making a tool that lets you search many Elixir packages from a single website. Yeah, it’s a website and not a package (couldn’t find a better section).
I’ll show you what I mean. This is very much a WIP, but i have the data in and search does function.
Here’s a few implementation details.
Data source
There are many ways to get data, but I wanted this to work with any package (not that all are supported) and without much going on on disk (don’t want to generate docs myself).
I have a spider that
- Gets the
sidebar-items.jsonfile from hexdocs.pm/{package} - Parses JSON, convert it into rows
- Inserts into DB
Crawly was way too much overhead, would use HTTPoison and Floki next time. May refactor at one point.
Fuzzy search
I’m just using ILIKE in Postgres for now.
def list_fuzzy(str) do
search_str =
str
|> String.split(" ")
|> Enum.intersperse("%")
|> Enum.join()
|> then(&"%#{&1}%")
from(p in Page,
where: fragment("? ilike ?", p.search_body, ^search_str)
)
|> Repo.all()
end
Ideally the index would be client-side for instant results, but I’m not sure how to do fuzzy search in JS yet and I probably wont think about it unless the service gets slow.
Version 1.0 plans
The end result would ideally have
- iframe with hexdocs straight in it for instant results (not sure how possible)
- auto-bookmark popular pages
- keyboard navigation
But we’ll see. It’s just a fun side-project.
Feedback welcome
I am building it for myself, but obviously would love more users and ideas on what would be awesome.
Also looking for cool domain names. otherwise it’s going to be hexdocs-search.krister.ee - not so pretty. I’d go with hexdocs.search, but that’s not easy to aquire. TLD is supposed to exist, but not on my provider, Namecheap.
Trending in RFCs
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #performance













First 10 of 57 Posts
gregvaughn
I don’t want to take away your fun on a side project, but if it were me, I’d use
mix hex.docs fetchto cache all docs for my project’s deps locally, then runrg(ripgrep) in that directory for function names.dimitarvp
Yeah, that’s exactly what I do. I crafted a few very simple
ripgrepregex-ish expressions andalias-ed them in my~/.zshrcand then just do e.g.exdoc Enum.group_byin the terminal.…Actually it might be time to extend that script to use
open(Mac) /xdg-open(Linux) so as to just open the docs in a browser.gregvaughn
Yes, and you should blog it and/or gist it for sharing!
dimitarvp
Yeah, probably before hitting 50 years old would be nice
(currently 42).
stefanchrobot
TIL
mix hex.docs fetch! It’s exactly what I was looking for!But… it would be great to have something like
mix hex.docs offline(without more args) that would open up aggregated docs for the dependencies of the current project. I was actually thinking about opening an issue on ExDoc to suggest something like that. Thoughts?Exadra37
Like this:
stefanchrobot
Yes, but one that allows easy jumping between offline docs.
cmo
I believe dash can do hex docs.
KristerV
I don’t mind. I thought about this approach, but it seems to me that the complexity outweighs the benefits here. btw i want the search to reflect the menu items and not just the functions, since guides and sections can be very useful. so it makes more sense to just parse the JSON that the sidebar is compiled of. No disk management, no grepping, no mistakes. Or is my logic flawed?
I’d love to see that too.
For Linux and Windows there’s also Zeal, but I couldn’t get used to the interface. Also there’s only one broken repo that transforms hexdocs into docsets, which Zeal needs, so only Elixir itself is supported.
LostKobrakai
I was toying yesterday with porting the code to a ex_doc formatter (similar to the epub one), so they could be generated directly with ex_doc and possibly hosted by hexdocs.