ExLLM - Multi-provider LLM library (inspired by Python’s LiteLLM)

I started this a couple weeks ago using Claude Code. I am a relatively experienced developer, but a newbie with Elixir and unfamiliar with standard conventions. This has not been fully reviewed line-by-line, but it’s in a mostly usable state so I thought I’d share it.

I have used this alongside my other new library - ExMCP (yeah, I’m terrible at naming) to build out a chat client similar to Block’s Goose. It’s been tested fairly extensively with Anthropic, OpenAI, and Ollama, but less with the others.

Feature List:

• Unified API: Single interface for multiple LLM providers
• Streaming Support: Real-time streaming responses with error recovery
• Cost Tracking: Automatic cost calculation for all API calls (if cost data is available)
• Token Estimation: Heuristic-based token counting for cost prediction
• Context Management: Automatic message truncation to fit model context windows
• Session Management: Built-in conversation state tracking and persistence
• Structured Outputs: Schema validation and retries via Instructor integration
• Function Calling: Unified interface for tool use across providers
• Model Discovery: Query and compare model capabilities across providers
• Capability Normalization: Automatic normalization of provider-specific feature names
• Error Recovery: Automatic retry with exponential backoff and stream resumption
• Mock Testing: Built-in mock adapter for testing without API calls (use with Response Caching)
• Response Caching: Cache real provider responses for offline testing and cost reduction
• Type Safety: Comprehensive typespecs and structured data
• Configurable: Flexible configuration system with multiple providers
• Extensible: Easy to add new LLM providers via adapter pattern

Currently supported providers:

• Anthropic
• OpenAI
• Gemini
• OpenRouter
• Groq
• xAI/Grok

Local providers:

• Ollama
• LM Studio
• Bumblebee

Please give feedback!

https://github.com/azmaveth/ex_llm