azmaveth
ExLLM - Multi-provider LLM library (inspired by Python’s LiteLLM)
I started this a couple weeks ago using Claude Code. I am a relatively experienced developer, but a newbie with Elixir and unfamiliar with standard conventions. This has not been fully reviewed line-by-line, but it’s in a mostly usable state so I thought I’d share it.
I have used this alongside my other new library - ExMCP (yeah, I’m terrible at naming) to build out a chat client similar to Block’s Goose. It’s been tested fairly extensively with Anthropic, OpenAI, and Ollama, but less with the others.
Feature List:
- Unified API: Single interface for multiple LLM providers
- Streaming Support: Real-time streaming responses with error recovery
- Cost Tracking: Automatic cost calculation for all API calls (if cost data is available)
- Token Estimation: Heuristic-based token counting for cost prediction
- Context Management: Automatic message truncation to fit model context windows
- Session Management: Built-in conversation state tracking and persistence
- Structured Outputs: Schema validation and retries via Instructor integration
- Function Calling: Unified interface for tool use across providers
- Model Discovery: Query and compare model capabilities across providers
- Capability Normalization: Automatic normalization of provider-specific feature names
- Error Recovery: Automatic retry with exponential backoff and stream resumption
- Mock Testing: Built-in mock adapter for testing without API calls (use with Response Caching)
- Response Caching: Cache real provider responses for offline testing and cost reduction
- Type Safety: Comprehensive typespecs and structured data
- Configurable: Flexible configuration system with multiple providers
- Extensible: Easy to add new LLM providers via adapter pattern
Currently supported providers:
- Anthropic
- OpenAI
- Gemini
- OpenRouter
- Groq
- xAI/Grok
Local providers:
- Ollama
- LM Studio
- Bumblebee
Please give feedback!
Where Next?
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Master modern tools, write cleaner code, and tackle real-world challenges with Manning books, videos, and courses. Start building today.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #ai
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixirconf-us
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Master modern tools, write cleaner code, and tackle real-world challenges.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 1- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
gtcode
This is great, because DSPy uses LiteLLM. Will switch out to using this lib for the LM adapter in DSPEx.
The Gemini integration looks comprehensive:
https://github.com/azmaveth/ex_llm/tree/master/test/ex_llm/adapters/gemini