Dexter - A fast, full-featured Elixir LSP optimized for large codebases

Your Libraries & OS Mentoring Libraries
,
1

Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It supports a slew of LSP features like go-to-definition, go-to-references, hover docs, near instant formatting, renaming of modules, functions, and variables across the codebase, and more.

Remote’s Elixir codebase is enormous (over 57k files) so we have run into every bottleneck you can imagine with the existing LSPs. Expert works great on smaller codebases, but it can’t yet handle codebases of our size. On our codebase, we’re usually not even able to get it to finish indexing.

Dexter started as a CLI tool I wrote for myself to improve my search keybindings for jumping around files due to not having a working LSP. I quickly realized that I had built the foundations for an LSP without the server. So I added an LSP server in front and Dexter was born. I released Dexter internally at Remote last week and the feedback has been very positive, so I wanted to open source it to help others with similar issues and give back to the community.

Dexter takes a different approach than the other Elixir LSPs. Rather than attempting to compile the codebase and glean info from the compiled code and parsed AST, Dexter simply parses the code of your project and dependencies directly. I was surprised by how fast and accurate this approach can be.

Everything in Dexter is built for speed without sacrificing correctness. There are limitations to the pure text-based approach, but we’re able to work around those with clever parsing and deferring any complicated macro/import injection tracing to runtime.

I know that the Elixir community has consolidated around Expert, which I think has been a great move. Dexter takes a different approach to the compilation-based LSPs to solve the specific pains we’ve had at Remote with such a large Elixir codebase, so I imagine it will be very helpful for those working on large codebases (although it works great on small codebases too!). While not everything is applicable, I’d imagine that there are some optimizations that we did in Dexter that could be similarly considered in Expert. But that being said, we’re really enjoying using Dexter at Remote and plan to continue maintaining and improving it into the future.

Give it a try and let me know what you think!

To install (assuming you have SQLite installed, if not install it first):

mise plugin add dexter https://github.com/remoteoss/dexter.git && mise use -g dexter@latest

And then just configure your editor, either with the VS Code/Cursor extension, NeoVim, or your other favorite editor.

If you have any thoughts, feature requests, or bug reports, I’d love to hear them. Likewise, contributions are absolutely welcome to the project.

Features

  • Fast indexing — cold index completes in ~11s on a 57k-file Elixir monorepo, ~100ms on Oban, ~300ms on the Elixir standard library (measured on an M1 MacBook Pro). After your first index, incremental indexing makes sure that you never have to reindex the whole codebase again.
  • Go-to-definition — jump to any module, function, type, or variable definition. Resolves aliases, imports, defdelegate chains, use injections, and the Elixir stdlib. Handles all definition forms: def, defp, defmacro, defprotocol, defimpl, defstruct, and more.
  • Go-to-references — find all usages of a function or module across the codebase, including through import, use chains, and defdelegate.
  • Hover documentation — @doc, @moduledoc, @typedoc, and @spec annotations rendered as Markdown when you hover over a symbol.
  • Autocompletion — modules, functions, types, and variables with full snippet support. Resolves through aliases, imports, use injections, and the Elixir stdlib. Works for qualified calls (MyApp.Repo.|), bare function calls, and module prefixes.
  • Rename — rename modules, functions, and variables with automatic file renaming when the convention is followed.
  • No compilation required — the index is built by parsing source files directly, not by compiling your project. Dexter works immediately on any codebase, even ones that don’t compile.
  • Monorepo and umbrella support — a single index at the repository root covers all apps and shared libraries. Go-to-definition, find references, and rename work cross-project out of the box.
  • Format on save — formats .ex, .exs, and .heex files on save via a persistent Elixir process. Near-instant after the first save. Formatter plugins (Styler, Phoenix.LiveView.HTMLFormatter) are loaded from your project’s _build — no install needed. Syntax errors are surfaced as diagnostics.
  • Elixir stdlib indexing — jump to Enum, String, Mix, and other bundled modules by indexing your local Elixir installation sources.
  • Signature help — parameter hints as you type function calls.
  • Workspace symbols — search for any module or function across the entire codebase.
  • Call hierarchy — navigate incoming and outgoing calls.
  • Code actions — add missing aliases with a single action.
  • Document symbols — outline view of all functions and modules in the current file.
  • Document highlight — highlight all occurrences of the symbol under the cursor.
  • Variable support — go-to-definition, rename, and completion for local variables via tree-sitter, with correct scoping across case, with, for, and other block constructs.
  • Git branch switch detection — automatically reindexes when you switch branches.

Checkout the GitHub Repo for more details:

84 Likes
2

This is fantastic.

Any chance of upstreaming some of Dexter’s code to Expert? Probably both projects can benefit from each other.

4 Likes
3

Dexter is so fast! At first I was like: huh, it indexed the project already? It must be broken!

But no, it just works. Try it!

5 Likes
5

Thank you! The code itself is probably not possible to upstream directly because it’s mostly written in Go (a decision I made when I wanted a fast CLI tool and didn’t realize it would turn into a full LSP). However, I think some of the concepts and performance optimizations that Dexter does could potentially be worked into Expert to improve performance on large codebases.

5 Likes
6

This is AMAZING!

I’ve never been a big fan of LSPs and use my own home-rolled ast-grep+ripgrep-based tool for Vim9. It does context-aware lookup, find references, and hover (these are the only features I really care about). It actually works quite well on smaller/medium codebases though is certainly still clumsy and sometimes still slow even on smaller code-bases as it is running grep ever time (my naive use resolving can grind to a halt).

In any event, I love the text-based approach as it opens up the possibility for editor-specific implementations without necessarily needing an LSP running. I do realize that avoiding editor-specific implementations is pretty much the raison d’etre for LSP, but of course it comes with the cost of being beholden to the LSP API.

So I have a few questions:

  1. The example you give in the README for look is MyApp.Repo.get which seems to resolve to my_app/repo.ex when it’s actually implemented in Ecto itself. This is something I was trying to figure out and eventually gave up “for now” (lol… well, I got it sorta working but it was unusably slow). Is this something you’ve been thinking of tackling? Is this something people even care about?

  2. Any chance of exposing doc lookup (hover) through the CLI?

  3. I know you just released this and I’m not quite sure what this would look like, but any thoughts on making Dexter extendable? Part of the reason I go with my own thing is that whenever I want to add my own tokens for lookup I can just do it. For example, on projects using ex_machina I can gd on insert(:foo which will take me to def foo_factory. And of course the big one is that I can jump to def for a bunch of Ash stuff. Again, I’m not sure what this would look like as I haven’t inspected the source yet (although I’ve never written a line of Go).

Oh boy, that was a bit longer than I expected.

Anyway, congrats on the release and great work! I’m now going to spend the evening writing a vim9 wrapper for this that I assume no one but me is gonna use :sweat_smile:

EDIT: I’m realizing fetch docs likely requires LSP (sorry, it’s been a while).

2 Likes
7

Last week?? That is a very short internal → public release timeframe! Do you not sleep?

1 Like
8

Hi @sodapopcan, thank you!

I think you’ll find that Dexter is exactly what you’re looking for. It does this same type of text-based parsing, but is more syntax aware and accurate.

The example you give in the README for look is MyApp.Repo.get which seems to resolve to my_app/repo.ex when it’s actually implemented in Ecto itself.

This was a bad example and I have updated the readme. We actually do resolve through the use macro and go directly to the injected def get function in the Ecto dependency, I just never updated that section of the readme to reflect that. However, you’ll notice that in that injected function, there are actually no docs! That’s another reason why this was a bad example, since Dexter correctly returns no hover docs in this case, because there are none.

Any chance of exposing doc lookup (hover) through the CLI?

Absolutely! I’ll throw it on the list of features to add. Doc lookup is independent of the LSP actually. I will admit that most of my attention has been on the LSP side of things ever since it became a proper LSP, but I think having everything exposed via the CLI is also great for things like scripting or those like yourself who don’t want to run the LSP.

but any thoughts on making Dexter extendable? Part of the reason I go with my own thing is that whenever I want to add my own tokens for lookup I can just do it.

Yes, I actually have an idea in mind about doing this at some point, but more-so along the lines of exposing an API for mass code changes in large codebases. Or perhaps to add faster linter rules. I don’t have it fully formulated yet.

For your specific example, this is a great example of the limitation with text-based parsing. ExMachina is very macro heavy, and even their insert function isn’t defined directly: lib/ex_machina/strategy.ex:64. Dexter handles some complex macro use-cases for go-to-definition, but this is one case that I don’t think is worth the work to handle, especially since technically the definition would be that macro I shared above and not the place that you defined your factory.

However, I know you’re talking more about going from insert(:my_thing) to def my_thing_factory wherever the factory is defined. This is indeed a perfect example where some sort of extension would be great, because this is an ExMachina specific thing. Alternatively, we could just accept that ExMachina is very common and add special handling for ExMachina’s macros to go to the right factory definition. Or maybe this should be a code action. I’ll do a bit more thinking on this because it would indeed be nice to have.

Anyway, congrats on the release and great work! I’m now going to spend the evening writing a vim9 wrapper for this that I assume no one but me is gonna use :sweat_smile:

Thanks again! Have fun with it! I will say that for your day-to-day coding, you’ll get the best experience just hooking it up to your editor as an LSP. Dexter is very lightweight if resources are your concern, and we rely on the editor’s LSP implementation to handle things like automatically reindexing files that change inside or outside of your editor. You can work around this by manually reindexing when you change stuff, but at a certain point aren’t you just rebuilding an LSP client? :grinning_face: I say this as a very happy NeoVim user.

1 Like
9

I indeed spent many late nights working on this since last week - but it has been a lot of fun! I built the first prototype in a very long evening and then couldn’t stop adding features every chance I could the rest of the week and weekend when I realized what was possible.

2 Likes
10

I gotta give myself some credit in that mine is pretty syntax aware, it mostly falls down when it comes to more complex uses (and I have very light definition of “complex” there :sweat_smile:) Vim’s syntax engine coupled with searchpair() is actually a very powerful combo for parsing out tokens and scoping. Dexter is certainly better, of course and already getting my hands dirty!

and we rely on the editor’s LSP implementation to handle things like automatically reindexing files that change inside or outside of your editor.

I have to actually start reading through Dexter’s code but is that to say the actual token parsing is editor-specific? Like you don’t use the LSP for jump to def?

I think having everything exposed via the CLI is also great for things like scripting or those like yourself who don’t want to run the LSP.

:purple_heart:

You can work around this by manually reindexing when you change stuff, but at a certain point aren’t you just rebuilding an LSP client?

Yes and no, lol. These are easily taken care of with jobs and autocmds and everything else is just vimscript. I just love hacking on Vim plugins and having a CLI tool that populates a sqlite db gets me excited :smiley: Other having an LSP running for all the Elixir projects I always have open is part of the reason I don’t love them (though maybe they’ve improved there?). I’ve also just never got one successfully configured with Vim and there is very little chance I will ever become a NeoVim user for, uh, let’s say: “for reasons.” I am a very happy Vim user, though :slight_smile:

As for extensions that’s great you are already thinking about it! I kinda just blurted it out and don’t have anything concrete in mind as to how it would work since I need to understand exactly how Dexter works under the hood first. I’d say that for your own sanity you’d probably want to keep any community stuff as an extension, but that’s just me.

We actually do resolve through the use macro and go directly to the injected def get function in the Ecto dependency

I did try $ dexter lookup MyApp.Repo get before commenting and it came back with line 1 of lib/my_app/repo.ex. Perhaps it’s just a problem with the cli client?

11

I have to actually start reading through Dexter’s code but is that to say the actual token parsing is editor-specific? Like you don’t use the LSP for jump to def?

No, we do use the LSP for go-to-definition, sorry if that was confusing. I was talking about reindexing of changed files: the editor sends didChangeFiles to the LSP server when a file changes anywhere under the current project, even if it’s not in an open editor window. Dexter listens for that event and reindexes accordingly. That way we don’t need to do a whole separate file watcher setup since the editor’s LSP client already handles this.

I did try $ dexter lookup MyApp.Repo get before commenting and it came back with line 1 of lib/my_app/repo.ex. Perhaps it’s just a problem with the cli client?

Yeah, this works well in the editor via the LSP, so it looks like a CLI-specific issue. I’ll need to look into it, but I believe this issue is because we actually use the file context from where you call your lookup to trace through macros. One of the tricks is that we look through the file’s imports and uses and use that context to broaden the lookup. It works very quickly when you’re doing it in the editor. So in this case because the lookup has no additional context, we fall back to the definition file, which is lib/my_app/repo.ex. I think the easy fix here is requiring passing in the current file into the CLI when looking things up.

As I said, I haven’t tested the CLI as extensively as the LSP because I mostly use the LSP myself, so haven’t hit these bugs. However, it helps to have people like you who do use the CLI find these bugs, so thank you! I’ll add this to my list to fix, but feel free to open as issue too for tracking.

12

This is really cool. Thanks for creating this.

I had some leftover gpt-5.3-codex-spark. Used it to spike dexterity:

2 Likes
13

Yep, that makes sense! I do exactly this with my thing (although I’m limited to how deep I can recuse through uses due to using a slow language).

As an aside, I’m curious how you deal with a file like the generated MyAppWeb in Phoenix projects. My initial thought was I can just look for modules inside __using___ definitions to add them as lookup candidates, but of course when you have the pattern the Phoenix uses where the imported modules are all in regular functions returning quoted values, this completely throws a wrench into that.

Otherwise no rush on CLI fixes, at least not on my behalf. I’m mostly dropping down to SQL here. I’ve been working on and off on this other idea for a while now that is essentially projectionist.vim except it deals in module names as opposed to file names. Dexter is a massive helper for me here.

14

This is incredible work, thanks so much

2 Likes
15

For anyone interested - Nix derivation:

{
  buildGoModule,
  fetchFromGitHub,
  installShellFiles,
  lib,
  stdenv,
}:
let
  name = "dexter";
  version = "0.5.3";
  src = fetchFromGitHub {
    owner = "remoteoss";
    repo = name;
    tag = "v${version}";
    hash = "sha256-8JjxR7Q+4OgBSIgODxIEU/0mC+bPp9Nz7uCAjfn4HiY=";
  };
in
buildGoModule {
  pname = name;

  inherit version src;

  nativeBuildInputs = [ installShellFiles ];

  proxyVendor = true;

  postInstall = ''
    mv $out/bin/cmd $out/bin/dexter
  ''
  + (lib.optionalString (stdenv.buildPlatform.canExecute stdenv.hostPlatform) ''
    installShellCompletion --cmd dexter \
      --bash <($out/bin/dexter completion bash) \
      --fish <($out/bin/dexter completion fish) \
      --zsh <($out/bin/dexter completion zsh)
  '');

  vendorHash = "sha256-1mJ4HdDCsZl/g8F+L+NrW2ACuiHe2aSheJO/1XfKAb4=";

  meta = {
    description = "Fast implementation of Elixir language server in Go";
    mainProgram = "dexter";
    homepage = "https://github.com/remoteoss/dexter";
    license = lib.licenses.mit;
    platforms = lib.platforms.all;
  };
}
12 Likes
17

This looks very promising! Thank you OP.

Feature request though: typing Foo and then ctrl+space and have the autocompletion suggest Myapp.Stuff.Foo and on accept it keeps Foo but adds alias Myapp.Stuff.Foo in the first group of alias statements below the lexically-closest defmodule expression opening.

1 Like
18

Looks promising, but… why does an LSP use sqlite?

19

The filesystem is a terrible database. SQLite – or any in-OS-process embedded DB – is a much better fit.

3 Likes
20

To cache, or not to cache, that is the question:
Whether 'tis nobler in the mind to suffer
The delays and hanging of outrageous fortune
[…]

This guy Billy has answered nearly all the questions of new era.

21

Why do you need a database in the first place? You need to store the index on disk to not re-index the project every time. You can just dump to JSON and it would work perfectly

22

Dumping it to .txt would also work perfectly.

In both cases you lose the ability to query stuff via SQL.

Before you ask: have you invented and/or using a better query language?

1 Like