pnezis
Hi all,
I’m releasing probex, a small dependency-free CLI built for one purpose: coding agents working on Elixir burn a meaningful share of their tokens — and their wall-clock — just re-reading code by guesswork. probex removes that tax. The same task completes in fewer calls, with smaller payloads, and without the re-reads.
Where the tokens actually go
Watch an agent work on a large Elixir codebase and a pattern repeats all day:
grep -nto find where a function starts — grep can’t say where it ends;- guess a window:
sed -n '148,200p'; - the window clips the block (or drags in three neighbours), so read again, wider;
- repeat per function, per file, per task.
Every guessed window is paid for in tokens whether or not it was the right window, every miss is paid for twice, and every one of those calls is a full agent round-trip. Mining one week of our agent transcripts found 2,369 windowed reads with a median window of 54 lines — for answers that are usually a dozen lines, exactly bounded, and known to the parser all along.
probex asks the parser instead:
$ probex body lib/accounts.ex register_user/2
lib/accounts.ex:148-163 def register_user/2 [defmodule MyApp.Accounts]
def register_user(attrs, opts \\ []) do
...
end
Exact boundaries, so a heredoc containing end, a sigil containing a fake def, or a keyword-form body can’t produce a clipped read. And one call takes many files × many selectors, so what used to be a shell loop of N reads is one round-trip:
probex body lib/a.ex lib/b.ex changeset/2 valid?/1 # both blocks, in both files
probex body lib/ handle_info/2 # every match under a directory
probex outline big_test.exs --kind describe,test # exact TOC, no bodies
probex cover my_app --fun register_user/2 --body # did my new test go green?
The same economics drove every command: outline replaces “read the first 200 lines to orient”; directives replaces the header-window read; cover replaces the most reinvented wheel in our transcripts (agents writing one-off coverage-HTML parsers) with uncovered lines attributed to the functions that contain them, reconciling with mix test.coverage including :ignore_modules. Errors are budgeted too: a miss returns the corrected command or the list of what does exist, so the next call is the answer instead of another probe.
Reading commands parse, never compile — no mix, no deps, no app boot — so they work mid-refactor on files that don’t currently build.
How it was built (the part I find fun)
No feature came from a human wishlist. It started with me being annoyed at the token bill; the ask was one sentence — this is wasteful, improve it. Every decision after that was the agents’:
- The feature set was mined from the agents’ own shell transcripts — the most repeated waste became the founding commands.
- Nothing ships without measured demand: invocations are logged (opt-in,
PROBEX_LOGFILE),probex statsreports which commands and options get used and which never do, and an end-of-day skill mined each day’s sessions for gaps and proposed extensions. An early intuition-ranked roadmap got its top pick wrong; evidence has decided ever since. - Agents improve it for agents: feedback filed by the agents using it, fixes made by agents with the test suite as the north star, and
probex prompt --writeinstalls the instructions intoAGENTS.md/CLAUDE.mdfrom the binary itself — a test fails if a flag exists that the prompt doesn’t teach.
My role: ask once, run the loop, and pay for the tokens.
Install
git clone https://github.com/pnezis/probex.git
ln -s "$PWD/probex/probex" /usr/local/bin/probex
probex prompt --write # teach it to your agents
Status: alpha, but not experimental — extracted from an internal tool used daily by a fleet of agents against a production monorepo.
License: MIT
If you run agents against elixir projects, try it and bring back your usage log: the entire project is an argument that agent tooling should be chosen by the agents’ measured usage, and your evidence makes it better. 
Where Next?
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
At WyeWorks, we believe great software is built by combining technical excellence with genuine human connection
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 7- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
onnimonni
Did you compare the performance against: GitHub - DeusData/codebase-memory-mcp: High-performance code intelligence MCP server. Indexes codebases into a persistent knowledge graph — average repo in milliseconds. 158 languages, sub-ms queries, 99% fewer tokens. Single static binary, zero dependencies. · GitHub or other similar tools?
dimitarvp
Willing to give this a try.
Have you compared against GitHub - jahala/tilth: Smart(er) code reading for humans and AI agents. Reduces cost per correct answer by ~40% on average. Install: cargo install tilth -or- npx tilth · GitHub? It’s similar, not 100% but pretty close.
pnezis
@onnimonni Not benchmarked, because they’re for different steps of the loop. An indexer answers facts about code — where a symbol lives, who calls it, how modules connect — and its token savings come from returning graph facts instead of source.
probexserves the step after that: once you know where to look, an agent still has to read the code to change it, and that’s whatprobexbounds exactly — the body of one function, all clauses of a multi-clause one (one symbol, several disjoint ranges), a file’s outline with real end lines, the test block by title, the uncovered lines grouped by function.So it’s navigation vs. reading — they compose rather than compete. An indexer telling you
accounts.ex:148is exactly what feedsprobex body accounts.ex:148.If someone runs both in one agent loop I’d genuinely like to hear how it goes.
pnezis
I haven’t — honestly, I didn’t know about
tilthuntil this comment, so thanks for the pointer. It does look close, and the benchmark methodology (cost per correct answer) is exactly the right metric for this class of tool.The difference is a deliberate bet rather than a feature gap:
probexis Elixir-only, on the language’s own parser instead of a tree-sitter grammar. A generic grammar sees Elixir’s syntax; the compiler’s parser sees the language — and that’s what gives us the freedom to keep extending along the things that are unique to Elixir rather than common to 16 languages. Multi-clause functions as first-class citizens (one symbol, several disjoint ranges, addressable together or per clause), describe/test blocks by title, macro DSL blocks in routers and schemas,defstructwith its@derive/@enforce_keysrun, the preamble of a test file as a nameable thing. And it goes below source level where tree-sitter can’t follow: cover joinsmix test --cover databack to the functions that own each line, down to what the BEAM actually compiled — overridable renames, macro-injected functions.dimitarvp
Thank you, thought so.
tilthfirst, thenprobexindeed.Here’s what Fable has to say about
probexfor my work:Feedback from a first day of real use (Elixir 1.20/OTP 29), from an agent loop that already ran tilth for navigation:
defmoduleblocks are unreachable throughbody. Givendefmodule Grace do … defmodule CloseFacts do defstruct … end … end:probex body grace.ex defmodule:CloseFactsreports no match — while the tool’s own “no match for CloseFacts” hint suggests exactly that command.probex body grace.ex L58(a line inside CloseFacts, outside any function) errors “inside defmodule CloseFacts (37-75), not inside any function” instead of returning the enclosing module block.outlinelists the nested module with the right range, so the information is there. Struct-per-nested-module is a common Elixir layout (Types.Breakdown,Grace.CloseFacts), so this one hits daily.probex body coordinator.ex calculate_interest_state/3returnedcalculate_interest_state/4with exit 0 and nothing on stderr. Returning the near miss is useful, but it should say so (or exit 2 with the candidate) — per the README’s own “no silent no-ops” rule.test:/describe:titles are exact-only. Agents usually know a fragment of a long title; substring (or glob) matching, or listing near titles on a miss, would remove one round-trip.@golden_sha256 "…") are not addressable (no match, no list of what exists).preambleprints range1-17but labels it7 lines..eex/.heexgive “cannot parse — unexpected token” rather than an “unsupported extension” refusal.What worked exactly, for the record: name/arity bodies, multi-selector batches,
clauseson head-dispatch functions (and the ambiguity error pointing at it), keyword-form one-liners,L<n>/file:N,outline --kind describe,test,preamble,directives,--head,--decorators, andcoveragainst a realmix test --cover --export-coveragerun (per-function attribution, correct source). Payload on nine real reads: about 40% of the sed windows they replaced, zero clipped blocks.pnezis
@dimitarvp thanks a lot for fable’s feedback on
probex, most items are addressed and live on maindimitarvp
Thank you. Fable confirmed 5/6 fixed (after running tests on our codebase) and said only the substring/glob matching for
test:/describe:titles is left from its original batch of feedback.probexhas been hugely useful to me even for just a few days of usage; a real token reduction. And I don’t always care about token expenditure TBF; but the extra turns that the agents need to find exact code blocks really do take extra time. Withprobexthe amount of turns got reduced (not by much, maybe 3-5%, but having in mind that sometimes a turn can take 5-10 seconds, it does add up and the difference is felt).So, super thank you. Genuine improvement to the ecosystem.