quatermain
Hello,
I’m sharing my plugin here in forum after some time so it has time to mature and proof yourself.
I use Claude Code daily on a production Phoenix app (319K lines of Elixir). Claude is good, but it doesn’t know Elixir the way it needs to. So I built a plugin that enforces the rules it keeps breaking. This post is less about the feature list and more about the method behind it, because most of what I built first turned out to be dead weight, and I only know that because I measured it, across 1,800+ of my own sessions.
Why I built it
My main project is a production Phoenix app, some stats:
- ~ 319K lines of Elixir in lib,
- ~ 251K lines of tests,
- 77 contexts,
- 44 LiveViews,
- ~140 LiveComponents,
- 938 migrations.
At that scale you hit the edges of generic AI tooling quickly, token usage goes to sky, subscription to hell.
Claude will use :float for a money field. It will skip authorization in handle_event because mount already checked. It will query the database in mount and not care that mount runs twice. Not always. Sometimes the code is very good. That inconsistency is the actual problem, and I can’t fix the model.
What I can fix is everything around the model. Rules that stop known mistakes before they ship. Compile and test checks after every change. Review that doesn’t depend on my energy level at 11pm.
Jose Valim wrote on the Tidewave blog that the future of coding agents is vertical integration. That’s the bet here: a general agent plus deep Elixir rails beats a general agent alone.
The embarrassing part
v1.0.0 shipped on February with skills for LiveView, Ecto, Oban, OTP, testing and security. I was quite proud of it.
Then I analyzed 160 of my own sessions with ccrider by Neil Berkman (read-only MCP server that lets Claude read its own session history) and findings:
- Skill auto-loading fired zero times in 160 sessions. The knowledge existed. It was never delivered.
- My PostToolUse hooks were silently broken for a full month. Hook stdout goes to verbose mode only in Claude Code, so every security reminder and format check I wrote did nothing.
- I ran
mix compile --warnings-as-errorsmanually 11 to 27 times per session. Every session. The plugin was supposed to do that for me.
Anyone can generate a pile of skills in an afternoon, and reading them tells you nothing. Mine looked complete and never fired. You have to measure.
What it became
Five months and +30 releases later, the plugin is organized around one workflow:
brainstorm → plan → work → review → compound
The filesystem is the state machine. Plans are markdown files with checkboxes, progress survives session crashes, and the compound phase captures every solved problem as a searchable solution doc, so the same bug never gets investigated twice.
Enforcement lives in hooks, not prose. There are 26 Iron Laws that stop the code before it ships:
# Iron Law #4: NEVER use :float for money
field :price, :float # STOP, use :decimal or :integer
# Iron Law #11: AUTHORIZE in every handle_event
def handle_event("delete", %{"id" => id}, socket) do
delete_item(id) # STOP, where is the authorization?
end
# Iron Law #15: no implicit cross joins
from(a in A, b in B, select: {a, b}) # STOP, Cartesian product
Current state:
- 51 skills,
- 26 specialist agents,
- 139 reference docs,
- 31 hooks.
That is more surface area than I planned, and it only stays trustworthy because of the feedback loops below. Recent additions go beyond code generation: a Hex supply-chain audit (/phx:deps-auditchecks dependency tarballs for Trojan Source bidi characters, compile-time exec and typosquats), first-class Ash Framework support, and an optional cross-model review where OpenAI’s Codex acts as an external critic on Claude’s work.
The part I actually care about: feedback loops
The plugin improves through three loops, and this is what I would defend as the real work:
1. Session retrospectives. The pipeline kept running after those first 160 sessions: 400 more deep-analyzed in June, and by the July audit the corpus it sweeps had grown to 1,853 sessions (the earlier batches are subsets of it). The question is always the same: how did this skill actually behave in my real sessions? Did it fire, did it help, did I have to correct it afterwards? Some answers hurt:
- My CLAUDE.md routing rules had a 0% firing rate across all 400 deep-analyzed sessions. Pages of routing documentation I was proud of, and the model never acted on them once. Routing moved into hooks, which fire deterministically.
/phx:workhad a 0.61 correction rate. I was redirecting it in more than half of its runs. It now has to read the plan scratchpad and verify intent before touching code.- One orchestrator agent was spawned exactly zero times across the whole corpus. I rewrote it into a research fan-out the workflow actually uses.
2. Deterministic evals. Every skill is scored on 8 dimensions (structure, triggering, safety, clarity…) in CI. Every change must pass before it lands. This is also the tripwire for the next silent breakage, because after the hooks story I assume there will be one.
3. Trigger tournaments. Skill descriptions are tested against held-out prompts that real users type. Four weak skills went from 50.5% to 78% activation accuracy.
And one number I find funny: even with all this, the plugin actually gets used in about 16.5% of my own sessions. Most sessions simply don’t need it. I consider that correct behavior. Tooling that intervenes when you don’t need help is worse than no tooling.
A small thing that made my month: the author of the recent bluez library announcement here wrote that they’d mainly been building its features with this plugin, with manual and AI reviews on top. Seeing it help ship a real library was worth more to me than any star counter (486 currently), because star = one click, actual usage and telling others “I use it” require more effort than one click.
Honest limitations
- Orchestrated phases cost tokens.
/phx:fullspawns research, implementation and review agents./phx:quickexists for a reason. - It’s strongest with Tidewave running (live process state, runtime eval, SQL). Without it you get static analysis only or try to run mix run/eval
- The model stays the model. Iron Laws catch known mistake patterns. They don’t turn Claude into a senior Elixir engineer, and I still review everything it writes.
- Every number in this post is me measuring my own sessions on my own projects and mostly processed by Claude Code
Try it
/plugin marketplace add oliver-kriska/claude-elixir-phoenix
/plugin install elixir-phoenix
/phx:intro # guided tour, ~5 minutes
Site with docs:
Source code:
Where Next?
Trending in Announcing
Other Trending Topics
Latest Phoenix Threads
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
Enabling companies to succeed by building software people love.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All Posts (oldest first)
- Show All Posts (newest first)
rogerleite
I would just like to thank you.
I have been using this plugin for some time and it’s helping a lot. The plan → work steps are great, and the *brainstorm *helps me a lot to build artifacts with alternatives to discuss with the team.
The review command is also good, but I don’t use it all the time because of the token costs.
Just wanted to say thank you!
quatermain
Thanks for using, happy it’s helping people.
I can look into review, maybe we can achieve some light version. It’s already heavy optimized so be sure you have the latest version, in early days it was really huge hole for tokens. But still it starts multiple sub agents and sometimes it works for 30 minutes for bigger changes so …
rogerleite
Nice! I’m going to update it and try again to see if is using less tokens.
quatermain
Plugin got upgrade to V3 and now support Claude Code, Codex, AMP, Pi and OpenCode and some improvements.
See changelog
[3.0.0] - 2026-07-25
Major release: keep Claude Code as the full canonical plugin while adding
generated, runtime-native skills distributions for Amp, Codex, Pi, and
OpenCode. Each target is explicit about supported workflows and deferred
capabilities rather than claiming cross-runtime feature parity.
Added
LiveView client-boundary guidance — documents that
phx-value-*, form,and hook payloads are user-controlled; clarifies that exposed IDs are not an
authorization flaw by themselves and must be checked against server-side
state in every event.
Generated multi-runtime skills distributions — all 51 canonical skills,
complete resource subtrees, non-Markdown bytes, and executable modes now ship
as deterministic targets for Amp, Codex, Pi, and OpenCode. Target generators
normalize names and invocation syntax, validate references and manifests,
reject collisions and escaping resources, build through staging directories,
preserve prior output on failure, and provide read-only drift checks. Claude
Code remains the canonical source and retains its full native hooks, agents,
MCP, permissions, and instructions.
Canonical multi-runtime support matrix — documents native invocation,
distribution, supported and deferred capabilities, generated-target
acceptance requirements, and isolated Amp, Codex, Pi, and OpenCode smoke-test
contracts.
All-runtime generated skills sync command — contributors can run
make generated-skills-syncto regenerate and validate Amp, Codex, Pi, andOpenCode in sequence, then validate the reviewed golden snapshots.
Target-specific sync and validation commands remain the authoritative
implementation and keep failures attributable.
Codex native destructive-command safeguard — the generated Codex plugin
now includes one trust-gated, synchronous
PreToolUsehook that reuses thecanonical audited blocker for destructive Ecto operations, unguarded force
pushes, and accidental production Mix commands. The broad Claude hook set,
async hooks, custom agents, and Tidewave MCP remain deferred.
Optional runtime smoke harness —
make amp-runtime-smoke,make codex-runtime-smoke,make pi-runtime-smoke, andmake opencode-runtime-smokevalidate locallygenerated targets, native installation or discovery, all 51 installed skills,
retained resources and modes, and fresh-process removal in isolated temporary
homes without requiring credentials or making model calls.
Generated-target golden snapshots — CI now pins aggregate path, byte, and
executable-mode digests for Amp, Codex, Pi, and OpenCode before shared
generator refactoring. Target changes require an explicit reviewed snapshot
update instead of silently redefining the baseline.
Package-specific
/phx:learn-from-fixrouting — verified fixes andexplicit user-taught rules can now be saved as native background skills with
--library <package> --scope personal|project. The workflow reads lockedversions, checks both scopes for shadowing, merges safely, and never writes to
cached plugin files. General cross-project rules route to personal
~/.claude/CLAUDE.md. Description-based activation remains model-selected.Amp target drift pre-commit guard — canonical changes under
plugins/elixir-phoenix/skills/now triggermake amp-skills-validateandrequire regenerated
targets/amp/skills/changes to be staged. CI retainsthe same drift check as the authoritative fallback. Contributors can run
make amp-skills-syncto regenerate and verify the complete target in onecommand.
Changed
Optional release-service configuration — deployment guidance now gates
optional S3, Redis, and similar credentials behind their feature switch so
unrelated release commands such as
bin/migratecan start without them.Portable Amp investigation and review workflows — generated Amp
phx-investigateandphx-reviewnow preserve evidence-first investigation,read-only review, optional native workers, and complete same-session sequential
fallbacks without Claude task APIs, named agents, hooks, or MCP identifiers.
Portable PR-review/full workflow overlays — generated Amp, Codex, Pi, and
OpenCode
phx-pr-reviewnow use available GitHub connectors or authenticatedgh, preserve read-only triage and explicit mutation approval, and never inferreplies or resolution. Generated
phx-fullnow preserves user phase gates,bounded retries/cycles, explicit verification, read-only review, and compound
completion through portable skill invocation or sequential same-session
execution. Canonical Claude output remains unchanged; other workflows are not
claimed portable.
Portable plan/work workflow overlays — generated Amp, Codex, Pi, and
OpenCode
phx-planandphx-workskills now use scratchpad research checklists, plancheckboxes, and
progress.mdinstead of Claude-only named agents, task APIs,question tools, hooks, or MCP identifiers. Optional native subagents and
Tidewave retain complete same-session sequential fallbacks. Canonical Claude
skills are unchanged; this does not claim parity for other generated workflows.
Codex skill descriptions now preserve routing signal within a compact
budget — generated descriptions are capped at 120 characters while retaining
key capability and trigger cues. Route-sensitive skills keep explicit negative
routing rules to avoid collisions. This reduces pressure on Codex’s shared
skills context without changing canonical Claude descriptions, explicit skill
bodies, or other generated runtimes.
Amp installation no longer requires cloning this repository — the primary
project-local and global instructions now install all 51 generated skills
directly from the GitHub
targets/amp/skillstree. Update instructions alsoclarify that Amp copies skills and requires an explicit
--overwriteinstall.Fixed
Examples skill routing — clarify that requests for sample code, proper
implementations, walkthroughs, and expected workflow output should load the
examples skill alongside the relevant domain skill.
Accurate Tidewave MCP setup boundary — update the maintained Phoenix
dependency requirement and repository link, distinguish exposing Tidewave’s
HTTP server from registering it with a client, and document external
registration for Claude Code, Codex, and OpenCode.
Claude Code 2.1.217–2.1.220 nested-agent compatibility — flagship
workflows account for the depth-1 default in 2.1.217–2.1.218 and the restored
depth-3 default in 2.1.219+. Explicit lower-depth configurations keep
orchestration in the main conversation while spawning leaf specialists
directly, preserving the plan/work/verify/review contracts. Nested
investigation tracks apply tracing directly to avoid a depth-4 chain.
Claude Code 2.1.212 Agent invocation compatibility — removed the ignored,
deprecated Task/Agent
modeparameter from workflow instructions. Subagentsinherit the parent session’s permission mode.
Fork-session continuity on Claude Code 2.1.214+ — resume, scratchpad, and
branch-freshness SessionStart checks now also run for source
fork.Claude Code 2.1.218 agent validation compatibility — declared agent names
remain unqualified while concrete runtime invocations use the
phx:pluginnamespace. Regression coverage rejects colons in declared agent names and
unrecognized agent tool names.
Claude Code slash-command compatibility remains stable in v3 — the
marketplace package keeps its
elixir-phoenixinstall identity while theplugin namespace now explicitly registers
/phx:*; auto-installedectoand
lvcompatibility plugins preserve/ecto:*and/lv:*. The calltracing skill directory now matches
/phx:trace, and regression coveragechecks effective namespace-plus-frontmatter command names. Canonical
frontmatter names contain only the final command segment because Claude Code
2.1.216+ retains the plugin prefix when a skill declares
name.v2 → v3 upgrade path preserves newly added compatibility dependencies —
install the new
ectoandlvcompatibility plugins from the updatedmarketplace before updating
elixir-phoenix. This avoids a temporarymissing-dependency state on existing installations. Restart Claude Code
before relying on the updated hooks and agents.
Generated freeze is now honest about its safety boundary — Amp, Codex,
Pi, and OpenCode receive an advisory current-session edit scope instead of a
sentinel that falsely claimed enforcement by an uninstalled Claude hook.
Generated nested resource links now resolve from their containing file —
Amp, Codex, Pi, and OpenCode projections compute skill-relative paths from
each Markdown resource directory instead of the skill root, fixing broken
cross-skill and sibling links under nested
references/directories.Pre-release test and docs-site consistency —
npm testnow includes thescripts/tests/port suite, matchingmake testand CI, and the docs-siterebuild workflow triggers on all five runtime guides instead of only
docs/amp.md.Pi Git-package metadata now matches the generated distribution — the
repository-root manifest used by the documented Pi installation reports the
same
pi-elixir-phoenixname, 3.0.0 version, and Pi-focused description astargets/pi/package.json.Amp generation now rejects source symlinks and preserves Markdown modes —
generated resources can no longer dereference links outside the canonical
skill tree, and transformed Markdown permissions no longer depend on the
invoking process’s umask.
Runtime support and security claims now match operational boundaries —
the flagship portability contract is scoped to adapted Codex, Pi, and
OpenCode workflows; Codex documents and surfaces its Bash/
jqhookprerequisites; and security documentation distinguishes absent telemetry or
covert egress from visible, user-authorized external service calls.
Codex dependency-vetting routing metadata stays complete — the generated
phx-deps-vetdescription now preserves both its post-audit trigger and itsdistinction from dependency scanning instead of ending mid-clause.
Reliable behavioral trigger gates and routing boundaries — deterministic
structural evals no longer depend on ignored local result caches, while
make eval-fullruns a fresh Claude Haiku 4.5 gate requiring every skill toreach 75% accuracy. The judge now receives complete descriptions, validates
the exact skill/fixture set, retries infrastructure failures, rejects malformed
output, and writes results atomically. Generalized routing fixes brought all
51 skills above the threshold in the final full run (95.74% aggregate).
Generated-runtime install lifecycle guidance — Amp now documents native,
target-scoped removal and exact-sync boundaries; Pi distinguishes configured
Git refs from unpinned dependencies and includes both project and user clean
reinstall flows; OpenCode uses its documented plural skills path and treats
slash invocation as a tested 1.17.2 convenience rather than the portable API.
Codex plugin skill references now use their required runtime namespace —
explicit invocations and generated sibling references use
$elixir-phoenix:phx-investigaterather than the non-resolving unqualified$phx-investigateform. README and Codex installation guidance now documentthe exact plugin-qualified syntax inserted by
/skills.Portable command rewriting now requires complete invocation tokens — Amp,
Codex, Pi, and OpenCode projections leave filesystem paths, URLs, uppercase
names, underscored suffixes, and malformed namespace wildcards unchanged. Pi
now emits native
/skill:*syntax directly instead of passing through apath-like intermediate form.
Learning destination and skill-loading documentation — project-keyed
auto-memory is no longer described as applying to every Elixir project, and
the intro now documents native
paths:frontmatter as a file-path gate.quatermain
@AstonJ can you please update title of this post? Plugin now support Claude Code, AMP, Codex, Pi, OpenCode and not only Claude Code.
Thanks
AstonJ
You should be able to do it Oliver - is the edit option (pencil icon at the bottom of the post) not showing for you?
quatermain
Hi,
I’m not able to edit title
AstonJ
Ah right, I thought the setting would allow title edits too. Edited it for you - just let us know if you’d prefer something different (we usually use the format: Name - short description)
quatermain
wonderful, thanks. It’s perfect.
quatermain
Small beta update, for AMP user there is AMP’s plugin in development which allow to select specific skills from main plugin.