Alike 
- Semantic similarity testing for Elixir
Hey everyone! I’m excited to share Alike, a library for testing semantic similarity in Elixir.
The Problem
Testing LLM outputs, chatbots, or NLP pipelines is tricky. String matching is too brittle:
# This fails even though the meaning is the same
assert response == "Hi there! How can I help you?"
# actual: "Hello! How may I assist you today?"
The Solution
Alike introduces the wave operator <~> for semantic similarity assertions:
import Alike.WaveOperator
# These pass - same meaning, different words
assert "The cat is sleeping" <~> "A feline is taking a nap"
assert "Hello! How can I help?" <~> "Hi there! What can I do for you?"
# This fails - contradiction detected
refute "The sky is blue" <~> "The sky is red"
The magic of ML in your test suite
What I find incredible about building this in Elixir is that we can run actual ML models inside our test suite, as naturally as any other code.
Thanks to Bumblebee and Nx, the models run as supervised Nx.Serving processes. They’re just GenServers! This means:
- Parallel tests just work - With async: true, multiple test processes call the model concurrently, and Nx.Serving batches requests automatically
- No external services - No Docker containers, no Python sidecars, no API calls. The models live in your BEAM instance
- One model, many callers - Unlike Python where parallel test workers each need their own model copy (or a separate inference server), Elixir shares one model across all test processes efficiently
It’s the kind of thing that feels like it shouldn’t be this easy, but the BEAM + Nx ecosystem makes it natural.
How it works
Alike runs two models locally via Bumblebee + Nx:
- Sentence embeddings (all-MiniLM-L12-v2) for similarity
- NLI model (nli-distilroberta-base) for contradiction detection
No API keys needed - everything runs on your machine.
Installation
def deps do
[{:alike, "~> 0.2.0", only: :test}]
end
Links
- Hex: alike | Hex
- Run - Livebook.dev
- GitHub:
Would love to hear your feedback!
