azmaveth

azmaveth

ExLLM - Multi-provider LLM library (inspired by Python’s LiteLLM)

ExLLM - Multi-provider LLM library (inspired by Python’s LiteLLM)

I started this a couple weeks ago using Claude Code. I am a relatively experienced developer, but a newbie with Elixir and unfamiliar with standard conventions. This has not been fully reviewed line-by-line, but it’s in a mostly usable state so I thought I’d share it.

I have used this alongside my other new library - ExMCP (yeah, I’m terrible at naming) to build out a chat client similar to Block’s Goose. It’s been tested fairly extensively with Anthropic, OpenAI, and Ollama, but less with the others.

Feature List:

  • Unified API: Single interface for multiple LLM providers
  • Streaming Support: Real-time streaming responses with error recovery
  • Cost Tracking: Automatic cost calculation for all API calls (if cost data is available)
  • Token Estimation: Heuristic-based token counting for cost prediction
  • Context Management: Automatic message truncation to fit model context windows
  • Session Management: Built-in conversation state tracking and persistence
  • Structured Outputs: Schema validation and retries via Instructor integration
  • Function Calling: Unified interface for tool use across providers
  • Model Discovery: Query and compare model capabilities across providers
  • Capability Normalization: Automatic normalization of provider-specific feature names
  • Error Recovery: Automatic retry with exponential backoff and stream resumption
  • Mock Testing: Built-in mock adapter for testing without API calls (use with Response Caching)
  • Response Caching: Cache real provider responses for offline testing and cost reduction
  • Type Safety: Comprehensive typespecs and structured data
  • Configurable: Flexible configuration system with multiple providers
  • Extensible: Easy to add new LLM providers via adapter pattern

Currently supported providers:

  • Anthropic
  • OpenAI
  • Gemini
  • OpenRouter
  • Groq
  • xAI/Grok

Local providers:

  • Ollama
  • LM Studio
  • Bumblebee

Please give feedback!

https://github.com/azmaveth/ex_llm

Most Liked

gtcode

gtcode

This is great, because DSPy uses LiteLLM. Will switch out to using this lib for the LM adapter in DSPEx.

The Gemini integration looks comprehensive:

https://github.com/azmaveth/ex_llm/tree/master/test/ex_llm/adapters/gemini

Where Next?

Popular in Announcing Top

josevalim
Hello everyone, We have just released NimbleCSV which is a small and fast CSV parsing library for Elixir. It allows developers to define...
New
Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Thank...
New
Crowdhailer
I have been updating a library that allows you to pipe between functions that use the erlang result tuple convention. Assuming you have ...
New
bryanjos
Hi, I just published version 0.23.0 of Elixirscript. https://github.com/bryanjos/elixirscript/blob/master/CHANGELOG.md Most of the chan...
New
handnot2
Samly can be used to enable SAML 2.0 Single Sign On in a Plug/Phoenix application. This library uses Erlang esaml to provide plug enabl...
New
ityonemo
Currently just starting out on a new mini-project - getting zig NIFs to run in elixir. https://github.com/ityonemo/zigler The idea here...
New
maltoe
Hello! Came here to announce ChromicPDF, a pet project PDF generator I’ve been working on for the past few months. Why another PDF gener...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
kevinlang
Hey all, We have made an Ecto3 Adapter for SQLite3, ecto_sqlite3! We have successfully on-boarded the full suite of integration tests (...
New
tfwright
After working on it for a couple of months and using it in production for most of that time, today I’ve released LiveAdmin, a LiveView ba...
New

Other popular topics Top

vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

We're in Beta

About us Mission Statement