KristerV

How I currently use Hexdocs

I use hexdocs all day every day, but finding the right module and function takes too long even with my bookmark system.

Too much scrolling, reading, clicking around for the right function. I already know what function I need!

Solution

So I’m making a tool that lets you search many Elixir packages from a single website. Yeah, it’s a website and not a package (couldn’t find a better section).

I’ll show you what I mean. This is very much a WIP, but i have the data in and search does function.

Here’s a few implementation details.

Data source

There are many ways to get data, but I wanted this to work with any package (not that all are supported) and without much going on on disk (don’t want to generate docs myself).

I have a spider that

Gets the sidebar-items.json file from hexdocs.pm/{package}
Parses JSON, convert it into rows
Inserts into DB

Crawly was way too much overhead, would use HTTPoison and Floki next time. May refactor at one point.

Fuzzy search

I’m just using ILIKE in Postgres for now.

def list_fuzzy(str) do
    search_str =
      str
      |> String.split(" ")
      |> Enum.intersperse("%")
      |> Enum.join()
      |> then(&"%#{&1}%")

    from(p in Page,
      where: fragment("? ilike ?", p.search_body, ^search_str)
    )
    |> Repo.all()
  end

Ideally the index would be client-side for instant results, but I’m not sure how to do fuzzy search in JS yet and I probably wont think about it unless the service gets slow.

Version 1.0 plans

The end result would ideally have

iframe with hexdocs straight in it for instant results (not sure how possible)
auto-bookmark popular pages
keyboard navigation

But we’ll see. It’s just a fun side-project.

Feedback welcome

I am building it for myself, but obviously would love more users and ideas on what would be awesome.
Also looking for cool domain names. otherwise it’s going to be hexdocs-search.krister.ee - not so pretty. I’d go with hexdocs.search, but that’s not easy to aquire. TLD is supposed to exist, but not on my provider, Namecheap.

57 comments

#search #hexdocs #library-rfc

18 6178 57

2025-01-19 09:21:23 UTC

Most Liked

Show Best Posts
Show All (oldest first)
Show All (newest first)

gregvaughn

I don’t want to take away your fun on a side project, but if it were me, I’d use mix hex.docs fetch to cache all docs for my project’s deps locally, then run rg (ripgrep) in that directory for function names.

Post #1

stefanchrobot

TIL mix hex.docs fetch! It’s exactly what I was looking for!

But… it would be great to have something like mix hex.docs offline (without more args) that would open up aggregated docs for the dependencies of the current project. I was actually thinking about opening an issue on ExDoc to suggest something like that. Thoughts?

Post #5

dimitarvp

Yeah, that’s exactly what I do. I crafted a few very simple ripgrep regex-ish expressions and alias-ed them in my ~/.zshrc and then just do e.g. exdoc Enum.group_by in the terminal.

…Actually it might be time to extend that script to use open (Mac) / xdg-open (Linux) so as to just open the docs in a browser.