Dexter - A fast, full-featured Elixir LSP optimized for large codebases

This is fantastic.

Any chance of upstreaming some of Dexter’s code to Expert? Probably both projects can benefit from each other.

Dexter is so fast! At first I was like: huh, it indexed the project already? It must be broken!

But no, it just works. Try it!

Thank you! The code itself is probably not possible to upstream directly because it’s mostly written in Go (a decision I made when I wanted a fast CLI tool and didn’t realize it would turn into a full LSP). However, I think some of the concepts and performance optimizations that Dexter does could potentially be worked into Expert to improve performance on large codebases.

This is AMAZING!

I’ve never been a big fan of LSPs and use my own home-rolled ast-grep+ripgrep-based tool for Vim9. It does context-aware lookup, find references, and hover (these are the only features I really care about). It actually works quite well on smaller/medium codebases though is certainly still clumsy and sometimes still slow even on smaller code-bases as it is running grep ever time (my naive use resolving can grind to a halt).

In any event, I love the text-based approach as it opens up the possibility for editor-specific implementations without necessarily needing an LSP running. I do realize that avoiding editor-specific implementations is pretty much the raison d’etre for LSP, but of course it comes with the cost of being beholden to the LSP API.

So I have a few questions:

1. The example you give in the README for look is MyApp.Repo.get which seems to resolve to my_app/repo.ex when it’s actually implemented in Ecto itself. This is something I was trying to figure out and eventually gave up “for now” (lol… well, I got it sorta working but it was unusably slow). Is this something you’ve been thinking of tackling? Is this something people even care about?

2. Any chance of exposing doc lookup (hover) through the CLI?

3. I know you just released this and I’m not quite sure what this would look like, but any thoughts on making Dexter extendable? Part of the reason I go with my own thing is that whenever I want to add my own tokens for lookup I can just do it. For example, on projects using ex_machina I can gd on insert(:foo which will take me to def foo_factory. And of course the big one is that I can jump to def for a bunch of Ash stuff. Again, I’m not sure what this would look like as I haven’t inspected the source yet (although I’ve never written a line of Go).

Oh boy, that was a bit longer than I expected.

Anyway, congrats on the release and great work! I’m now going to spend the evening writing a vim9 wrapper for this that I assume no one but me is gonna use

EDIT: I’m realizing fetch docs likely requires LSP (sorry, it’s been a while).

Last week?? That is a very short internal → public release timeframe! Do you not sleep?

Hi @sodapopcan, thank you!

I think you’ll find that Dexter is exactly what you’re looking for. It does this same type of text-based parsing, but is more syntax aware and accurate.

The example you give in the README for look is MyApp.Repo.get which seems to resolve to my_app/repo.ex when it’s actually implemented in Ecto itself.

This was a bad example and I have updated the readme. We actually do resolve through the use macro and go directly to the injected def get function in the Ecto dependency, I just never updated that section of the readme to reflect that. However, you’ll notice that in that injected function, there are actually no docs! That’s another reason why this was a bad example, since Dexter correctly returns no hover docs in this case, because there are none.

Any chance of exposing doc lookup (hover) through the CLI?

Absolutely! I’ll throw it on the list of features to add. Doc lookup is independent of the LSP actually. I will admit that most of my attention has been on the LSP side of things ever since it became a proper LSP, but I think having everything exposed via the CLI is also great for things like scripting or those like yourself who don’t want to run the LSP.

but any thoughts on making Dexter extendable? Part of the reason I go with my own thing is that whenever I want to add my own tokens for lookup I can just do it.

Yes, I actually have an idea in mind about doing this at some point, but more-so along the lines of exposing an API for mass code changes in large codebases. Or perhaps to add faster linter rules. I don’t have it fully formulated yet.

For your specific example, this is a great example of the limitation with text-based parsing. ExMachina is very macro heavy, and even their insert function isn’t defined directly: lib/ex_machina/strategy.ex:64. Dexter handles some complex macro use-cases for go-to-definition, but this is one case that I don’t think is worth the work to handle, especially since technically the definition would be that macro I shared above and not the place that you defined your factory.

However, I know you’re talking more about going from insert(:my_thing) to def my_thing_factory wherever the factory is defined. This is indeed a perfect example where some sort of extension would be great, because this is an ExMachina specific thing. Alternatively, we could just accept that ExMachina is very common and add special handling for ExMachina’s macros to go to the right factory definition. Or maybe this should be a code action. I’ll do a bit more thinking on this because it would indeed be nice to have.

Anyway, congrats on the release and great work! I’m now going to spend the evening writing a vim9 wrapper for this that I assume no one but me is gonna use

Thanks again! Have fun with it! I will say that for your day-to-day coding, you’ll get the best experience just hooking it up to your editor as an LSP. Dexter is very lightweight if resources are your concern, and we rely on the editor’s LSP implementation to handle things like automatically reindexing files that change inside or outside of your editor. You can work around this by manually reindexing when you change stuff, but at a certain point aren’t you just rebuilding an LSP client? I say this as a very happy NeoVim user.

I indeed spent many late nights working on this since last week - but it has been a lot of fun! I built the first prototype in a very long evening and then couldn’t stop adding features every chance I could the rest of the week and weekend when I realized what was possible.

I gotta give myself some credit in that mine is pretty syntax aware, it mostly falls down when it comes to more complex uses (and I have very light definition of “complex” there ) Vim’s syntax engine coupled with searchpair() is actually a very powerful combo for parsing out tokens and scoping. Dexter is certainly better, of course and already getting my hands dirty!

and we rely on the editor’s LSP implementation to handle things like automatically reindexing files that change inside or outside of your editor.

I have to actually start reading through Dexter’s code but is that to say the actual token parsing is editor-specific? Like you don’t use the LSP for jump to def?

I think having everything exposed via the CLI is also great for things like scripting or those like yourself who don’t want to run the LSP.

You can work around this by manually reindexing when you change stuff, but at a certain point aren’t you just rebuilding an LSP client?

Yes and no, lol. These are easily taken care of with jobs and autocmds and everything else is just vimscript. I just love hacking on Vim plugins and having a CLI tool that populates a sqlite db gets me excited Other having an LSP running for all the Elixir projects I always have open is part of the reason I don’t love them (though maybe they’ve improved there?). I’ve also just never got one successfully configured with Vim and there is very little chance I will ever become a NeoVim user for, uh, let’s say: “for reasons.” I am a very happy Vim user, though

As for extensions that’s great you are already thinking about it! I kinda just blurted it out and don’t have anything concrete in mind as to how it would work since I need to understand exactly how Dexter works under the hood first. I’d say that for your own sanity you’d probably want to keep any community stuff as an extension, but that’s just me.

We actually do resolve through the use macro and go directly to the injected def get function in the Ecto dependency

I did try $ dexter lookup MyApp.Repo get before commenting and it came back with line 1 of lib/my_app/repo.ex. Perhaps it’s just a problem with the cli client?

I have to actually start reading through Dexter’s code but is that to say the actual token parsing is editor-specific? Like you don’t use the LSP for jump to def?

No, we do use the LSP for go-to-definition, sorry if that was confusing. I was talking about reindexing of changed files: the editor sends didChangeFiles to the LSP server when a file changes anywhere under the current project, even if it’s not in an open editor window. Dexter listens for that event and reindexes accordingly. That way we don’t need to do a whole separate file watcher setup since the editor’s LSP client already handles this.

I did try $ dexter lookup MyApp.Repo get before commenting and it came back with line 1 of lib/my_app/repo.ex. Perhaps it’s just a problem with the cli client?

Yeah, this works well in the editor via the LSP, so it looks like a CLI-specific issue. I’ll need to look into it, but I believe this issue is because we actually use the file context from where you call your lookup to trace through macros. One of the tricks is that we look through the file’s imports and uses and use that context to broaden the lookup. It works very quickly when you’re doing it in the editor. So in this case because the lookup has no additional context, we fall back to the definition file, which is lib/my_app/repo.ex. I think the easy fix here is requiring passing in the current file into the CLI when looking things up.

As I said, I haven’t tested the CLI as extensively as the LSP because I mostly use the LSP myself, so haven’t hit these bugs. However, it helps to have people like you who do use the CLI find these bugs, so thank you! I’ll add this to my list to fix, but feel free to open as issue too for tracking.

This is really cool. Thanks for creating this.

I had some leftover gpt-5.3-codex-spark. Used it to spike dexterity:

https://github.com/nshkrdotcom/dexterity