Osmotic - a self-hosted AI gateway for Anthropic and OpenAI (Phoenix)

Osmotic sits between your organisation and the model providers. It issues
API keys to people, teams, and applications, enforces token and USD
quotas, scans outbound requests for secrets, and records what every
request cost and who it belongs to. Keys carry attribution — person, team,
department, application — so cost reports survive re-organisations.
Phoenix + PostgreSQL, BSD 2-Clause, shipped as .deb packages — no
Erlang or Elixir on the host required.

Scope: Anthropic (Messages API + batch) and OpenAI (Responses API +
batch), each a native passthrough on its own listener and hostname, in
the provider’s own wire protocol. There is deliberately no unified
request schema — a parameter the abstraction does not model is a
parameter the client cannot use. Not in the box: Chat Completions,
Embeddings, Realtime, images, audio, fine-tuning; Google is on the
roadmap.

For Claude Code, adoption is one environment variable:

export ANTHROPIC_BASE_URL=https://gateway.example.com
export ANTHROPIC_API_KEY=pxy_…   # issued by the admin console

For the Codex CLI, a model_providers entry pointed at the OpenAI
hostname with wire_api = "responses". One proxy key works on either
hostname — a key identifies a person, not a provider.

The parts where Elixir did the heavy lifting:

• SSE frames are relayed as they arrive, byte-for-byte, no buffering.
  Usage is parsed out of the stream — never from a buffered body — and
  priced against an effective-dated rate card (including OpenAI’s
  service and context tiers), so a price change never rewrites history.
• Quotas run in alert or enforce mode; over an enforced cap the gateway
  answers 429 with Retry-After pointing at the billing-period rollover.
  One-off grants raise a cap for the current period and expire at
  rollover on their own.
• The DLP scanner is ReDoS-bounded and fails closed: if the pattern
  cache is unavailable, the request is refused, never silently
  forwarded.
• A daily job reconciles the gateway’s totals against both providers’
  cost APIs — Anthropic’s Cost Admin API and OpenAI’s organization Costs
  API — with age-aware drift classification, because provider figures
  get revised.

On method. I have not written production code by hand in years — the
career went technician → programmer → sysadmin → security officer →
DevOps/SRE → architect. The decisions in Osmotic are mine; the
implementation is Claude’s, held to CI gates (formatter, Credo, Sobelow,
gitleaks, tests) and verified against real provider billing. Questions
welcome — including the ones where I’ll have to read my own codebase
before answering.