georgeguimaraes
Hi all,
I want to show you Transmuter, a library that runs Hugging Face ONNX models on any Nx backend. (Nx is Elixir’s numerical library. ONNX is a open format to describe neural networks.)
Why
Bumblebee is great, but each architecture needs a port to Axon by hand, and Hugging Face gets new architectures and models weekly. Ortex runs ONNX models, but outside of Nx: you cannot use it in defn, and it has no Apple silicon GPU path.
Transmuter takes another approach. It implements the ONNX operators in Nx, one time each. Then it runs the graph of the exported model, operator by operator. So a new architecture needs no new Elixir code. It needs only operators that Transmuter already has, and Transmuter has all 203 operators of the ONNX standard domain, plus the ONNX Runtime contrib operators that Hugging Face exports use.
The result is plain Nx:
- It runs on EMLX (Apple silicon GPU) and on EXLA (CUDA or CPU).
- It works with
Nx.Serving. Nx.Defn.gradcan differentiate it.- Shapes are set when a function compiles, not when the model loads. So decoders with a KV cache and graphs with
Ifbranches work.
Models that Bumblebee can’t run yet
Bumblebee does not have these architectures today, and Transmuter can run them (I validated them):
- Vision: SAM, SAM 2 and the SAM 3 tracker, Depth Anything, Grounding DINO, OWL-ViT and OWLv2, Florence-2, DETR, YOLOS, RT-DETR and RF-DETR, Segformer, ORMBG, Swin2SR, SigLIP and SigLIP2, TrOCR and ViT-GPT2.
- Chats about images: SmolVLM, Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Ministral 3 and LLaVA.
- Text: GLiNER, Marian, Apertus, Granite 4.0-H (with Mamba-2 layers) and LFM2 (mixture of experts).
- Audio: wav2vec2, AST, CLAP, VITS, SpeechT5 and Kokoro.
Example
opts = [backend: EMLX.Backend, defn_options: [compiler: EMLX]]
serving = Transmuter.Text.text_embedding({:hf, "Xenova/all-MiniLM-L6-v2"}, opts)
Nx.Serving.run(serving, "The cat sat on the mat.")
#=> %{embedding: #Nx.Tensor<f32[384] ...>}
serving = Transmuter.Text.generation({:hf, "onnx-community/Qwen3-0.6B-ONNX"}, opts)
Nx.Serving.run(serving, %{messages: [%{role: "user", content: "What is the capital of France?"}]})
What you get
The servings follow Bumblebee where Bumblebee has the task:
- Text: embeddings, classification, reranking, zero-shot NER with GLiNER, NLI zero-shot, token classification, fill mask, question answering, generation (SmolLM2, SmolLM3, Qwen3, Gemma 3) and text to text (T5, Marian, NLLB, M2M100).
- Vision: classification, CLIP and SigLIP embeddings, captions (TrOCR, Florence-2), chats about images (SmolVLM, Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Ministral 3, Gemma 3, LLaVA), object detection (DETR, RT-DETR, RF-DETR), zero-shot detection (OWLv2, Grounding DINO), depth (Depth Anything), segmentation, background removal, SAM, SAM 2 and the SAM 3 tracker, and super resolution.
- Audio: Whisper with word timestamps, audio classification, CLAP zero-shot, and text to speech (VITS, SpeechT5).
Generation has greedy selection, sampling, beam search, logits processors, streaming and token log-probabilities.
Correctness
The tests compare the values with Python transformers pipelines and with ONNX Runtime on the same files. For the top Hugging Face export of each task, 75 of 77 repositories match. The README has the full coverage table, and it explains each model that does not match value by value.
Feedback
When a model has no ONNX file, you can export it one time with Optimum. If a model fails or gives other values than Python, please open an issue. That is the most useful feedback for me now.
- Docs: transmuter v2.0.0 — Documentation
- GitHub:
Trending in Announcing
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #umbrella
- #full-time-contract
- #elixirconf-us
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #elixirconf-eu
- #processes
- #security
- #api
- #forms
- #performance











Showing Posts 1 to 2- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
georgeguimaraes
btw, I have access to a RTX 3060 with 12GB of VRAM and my Macbook Pro with 16GB of RAM, so I’m a bit limited on which models I can run. If you can try bigger models, please do
tubedude
Nice! I tried in my Jev/Laya implementation and here is a summary:
It was good! But there is a 30 second compilation in
for each padding that I have to figure out how to fix.