yoavgeva
FerricStore - a durable, Redis-compatible cache
Hey everyone,
I’ve been working on something I want to share and get feedback on.
The itch
Every web app I build ends up with the same stack: a database, a cache layer, and the app itself. Three things to deploy, monitor, pay for, and hope they stay connected to each other.
For caching, the options are:
Redis — great protocol, great ecosystem, but in production you need a managed service (ElastiCache, Upstash, etc.) because self-hosted Redis persistence is… optimistic. AOF rewrite can lose your last few seconds. RDB snapshots lose everything since the last dump. And now you’re paying for a whole separate instance just to hold keys.
Java solutions* (Hazelcast, Infinispan) — powerful but painful to operate. Config files longer than your app code. JVM tuning. Cluster formation issues. Not exactly “easy to maintain.”
ETS/Mnesia — built into the BEAM, fast, but volatile. Node goes down, data’s gone. Mnesia replication exists but comes with its own set of surprises.
What I actually wanted: use the disk I’m already paying for on my instance, speak a protocol I already know (Redis), get real durability without a separate service, and keep my stack simple.
What I built with AI
[FerricStore]( GitHub - yoavgeva/ferricstore: Distributed crash-safe key-value store with Redis wire protocol. Durable by default — every write is Raft-committed and fsync'd. Embeddable Elixir library or standalone server. Built with Elixir + Rust NIFs. · GitHub ) is a persistent key-value store written in Elixir + Rust that:
Speaks RESP3 — connect with redis-cli, or any Redis client library that support RESP3. Your existing code mostly just works.
Every write is durable by default — Raft consensus + Bitcask append-only log + fsync. When you get OK back, your data is on disk. Not “eventually.” Not “if the AOF rewrite finishes.” On disk, right now.
Runs embedded in your app — add it as a dependency, FerricStore.set("key", "value"). No separate process, no network hop, no connection pool. Your cache lives in the same BEAM node as your app.
Or runs standalone — `docker run -p 6379:6379 yoavgeva/ferricstore` and connect with redis-cli. Drop-in for apps that already speak Redis.
50+ Redis commands — strings, hashes, lists, sets, sorted sets, TTL, MULTI/EXEC, pub/sub, and more.
Native Elixir commands — CAS (compare-and-swap), distributed locks, rate limiting, FETCH_OR_COMPUTE — things you’d build on top of Redis but get out of the box here.
Probabilistic data structures — Bloom filters, Cuckoo filters, Count-Min Sketch, TopK, HyperLogLog, T-Digest. All built-in.
Vector search — HNSW index for similarity search, built into the storage engine.
The architecture in 30 seconds
Client (redis-cli / Redix / your app)
|
v
RESP3 Parser (pure Elixir, zero-copy)
|
v
Command Dispatcher
|
v
Raft Consensus (via ra library — same one RabbitMQ uses)
|
v
Bitcask Storage Engine (Rust NIF — append-only log, CRC-checked)
|
v
ETS Hot Cache (recent values served from memory, cold values read from disk)
Writes go through Raft for consistency, then to Bitcask for persistence. Reads hit ETS first (microseconds), fall back to disk for cold data. The Rust NIF handles the low-level I/O — pure functions, no dirty schedulers, proper consume_timeslice yielding so the BEAM scheduler stays happy.
Embedded mode — the thing I’m most excited about
# mix.exs
{:ferricstore, "\~> 0.1"}
# your app
FerricStore.set("user:123:session", session_data)
{:ok, data} = FerricStore.get("user:123:session")
# with TTL
FerricStore.set("rate:api:123", "1", ttl: :timer.seconds(60))
# atomic operations
{:ok, new_count} = FerricStore.incr("page_views")
# compare-and-swap
:ok = FerricStore.cas("inventory:sku42", "10", "9")
No Redis connection. No connection pool config. No “what happens when Redis is down.” It’s just a function call that persists to disk. Your Phoenix app, your cache, one deployment, one thing to monitor.
Standalone mode — drop-in Redis replacement
If you have apps in other languages, or you just want a Redis-compatible server with real durability:
bash
# Docker
docker run -p 6379:6379 -v ferricstore_data:/data yoavgeva/ferricstore
# Then use any Redis client
redis-cli SET mykey "hello"
redis-cli GET mykey
Or from your Elixir app via Redix:
elixir
{:ok, conn} = Redix.start_link("redis://localhost:6379")
Redix.command!(conn, \["SET", "user:42", "alice"\])
Redix.command!(conn, \["GET", "user:42"\])
# => "Alice"
Everything you’d expect from Redis works — MULTI/EXEC transactions, pub/sub, pipelining, HELLO 3 (RESP3). Plus you get a built-in health endpoint (GET /health on port 6380), Prometheus metrics, ACL authentication, and TLS support.
The dashboard gives you live visibility into shards, key counts, memory pressure, hit rates, and slow queries — no Grafana setup needed.
What it’s NOT
Not a database replacement — it’s a cache/store. Great for sessions, rate limits, feature flags, job queues, counters, leaderboards. Not for your users table.
Not a sharded cluster (yet) — scales to 3-5 nodes via Raft replication for high availability and read scaling (every node serves reads from local ETS). Storage capacity scales with disk, not RAM — a 500GB NVMe gives you 500GB of cache. But there’s no hash-slot sharding across nodes yet, so every node holds all the data.
Not battle-tested in production — this is v0.1. It passes 8000+ tests including shard-kill recovery and multi-node cluster tests, but it hasn’t seen real production traffic yet. That’s where you come in.
Not going to beat Redis on raw throughput — Redis keeps everything in RAM and doesn’t fsync by default. But it’s persistent store where every write hits disk,
Why Elixir + Rust?
Elixir gives us the BEAM’s supervision trees, distribution primitives, and ETS for the hot cache. Rust gives us a memory-safe, zero-copy storage engine that doesn’t GC-pause or mess with the BEAM scheduler.
The NIF boundary is clean — Rust functions are pure and stateless. No Mutex, no shared state, no dirty schedulers. Just `v2_append_batch(path, entries)` and `v2_pread_at(path, offset)`.
Looking for
Feedback on the API, the architecture, the docs — what’s confusing, what’s missing, what would make you try it?
Early adopters willing to try it in a side project or staging environment and report what breaks.
Use case ideas — what would you use a durable, embedded Redis-compatible cache for in your Elixir apps?
Links:
- Hex: ferricstore | Hex
- Docs: ferricstore v0.5.7 — Documentation
- Docker: yoavgeva/ferricstore - Docker Image
- GitHub:
https://github.com/yoavgeva/ferricstore
Happy to answer any questions. And yes, the name is a pun — Ferric (iron/Rust) + Store.
First Post!
Most Liked
Asd
Hi, good library,
It’s a very big project (30k lines of Elixir code, 15k lines of Rust and around 100k lines of tests) implemented in only two weeks according to git history.
I am reading the code and it is… strange.
First of all, it is a cache, but it has strange eviction logic. If write too many keys (95% of keydir memory limit), the FerricStore will enter a keydir_pressure_level = :reject state, where it will fail all incoming set calls. Entering this state also triggers the eviction, but it will only be triggered asynchronously. And even after this eviction happens, I won’t be able to write new keys until the MemoryGuard process performs another check (that is for 100ms). I mean, it would make sense to block the writes, but it appears to just return :error and not pass any set further.
Then, its RAFT, which is strange given that your post says “no network hop”. The problem is that async writes do not check for leadership change. In the end, async means that cluster can be in an inconsistent state and one node may see different results from the other. Let me explain this, for example, node A performs async set("x", 1) and node B performs async set("x", 2). Async write is implemented in a way that you first write to ETS, then disk storage, then you fire-and-forget into RAFT cluster. But this fire-and-forget operation may not succeed (because leader changed or any other issue, its consensus after all). So, if this issue happens with the second write, you are going to end up in a situation where node A sees "x" = 1 and node B sees "x" = 2. That may be fine, but the worst thing is that this situation is never fixed. It makes this async_write not even eventually consistent. It is just inconsistent, forever. I’d expect async writes to be eventually completed if the system is alive.
Then, every time one shard log file collects more than 256MB of data (that value is hardcoded), it will do a persistent_term.put, triggering a global GC. Not a big deal, but a strange decision.
Then, Bitcask compaction is triggered in a scheduled way. It attempts to compact every 30 seconds, and at that time it performs multiple File.ls calls. I think that it is possible to check if compaction needs to be triggered when rotating the shard log file, when performing a check.
And yeah, whole idea to use bitcask is very strange. How bitcask works: it just appends new entries into file and then puts them in a big hash table like key -> log_file_name_and_offset. That’s it. You use an ETS as a hot cache alongside a bitcask. That is strange. Why not just use an append-only file (or files) and a one or two ETS tables (or :shards) which stores key -> value or log_file_name_and_offset. That’s minus 15k lines of Rust, no NIF required. In the end, its an IO-bound task and there’s no need to use NIF in the first place. If you did NIF for performance, I’d suggest optimizing the code and reviewing the performance manually. I suspect that Claude performed some flamegraph analysis on hot paths and optimized code for them (thats obvious with comments like “I do … in order to save 10 nanoseconds”, that made me laugh 
), but that resulted in benchmark cheating, where all expensive operations were moved to the background (like compaction), and were left without optimization.
Then, Sandbox code is spread around all the project, introducing lots of ifs in all the modules. I’d suggest to just start separate instances of the FerricStore in tests, one for each test. That would make the code much more simple.
In the end, congratulations on initial release! 
I don’t think that I will be using this project in its current state. I’d also suggest to not feed this review into Claude, because I think it would be much better if you used this review as a starting point in your own review of the code in order to learn more about database and caches. I can perform a more detailed review if you want (for a reasonable price of course).
I’d suggest checking out marvelous Nebulex. It has no built-in persistence, but you can use any storage as an adapter (or implement your own storage).
AstonJ
A few days ago we updated the flag system after having changed the post composition tip to the following a few weeks ago:
Type here. Use Markdown, BBCode, or HTML to format. Drag or paste images. Please use your own words (no AI slop please).
The flag system:
It’s AI Generated
Using AI as a smart dictionary, thesaurus, or translator is ok - using AI for content composition (or posting anything that looks like it) is not.
We’ve been monitoring AI generated posts in a log since Feb of last year and determined this is the best way forward for now.
If people want to post AI generated text/conversations, then the new section we created last year may be a good fit as it would ensure the flow of the main thread remains in line with expectations/the norm: https://forum.elixirforum.com/t/new-ai-conversations-section/72455 (we could probably update those guidelines now).
To the OP, you should be able to edit the post and submit a new version (just please make sure it’s in your own words - translated to English with the help of an AI is fine if need be).
ziinc
Benchmarks do not just “help with marketing”. You are asking busy developers to trust your solution on production projects, but won’t bother to do up even a simple Benchee benchmark to compare lookup speeds against existing libraries?
FWIW your library does target a very relevant use case for my day job, but I would not touch this with a 10 feet pole unless there is some effort made on code quality and performance assurance.
Some pointers:
- Your feature comparison table is against standalone caching components, which muddies the waters for your messaging on this as an Elixir library. There’s no comparisons against leading caching solutions like Cachex or Nebulex for example. Nor are there performance comparisons.
- Going through your readme without using AI is critical if you sent to win developer trust. The writing style is too indicative of AI tendencies without any critical editing. Your readme could be half the size with double the effectiveness in messaging.
- The standalone mode adds additional code and dependencies to the end user compilation, I would extract it out to a separate library if I were you.
If you want your caching solution to succeed in the Elixir space and community, it also has to address the concerns raised in such threads critically, as some have probably hand rolled similar solutions to what you are building in some shape or form.
Last Post!
yoavgeva
Yes, this is what I mean by durable here.
If any workflow command return success, the flow state was already written through FerricStore durable path before the ack.
So if the app / worker restart, the workflow is still there.
If FerricStore server restart, it should also recover the current flow state from the data dir / volume
and rebuild the hot indexes / projections from the durable state.
Important details:
- current flow state is the source of truth
- if worker dies while holding a lease, the flow is not lost
- after lease expire / reclaim, another worker can claim it again
- stale worker can’t complete old work because of lease / fencing tokens
- FerricFlow does not replay your handler code like Temporal
- external side effects still need idempotency keys
The model is durable workflow state (imagine persistence state machine), not durable code replay.
One caveat: you need to run FerricStore with persistent data dir / docker volume / disk. If you run on
tmpfs or delete the volume, of course there is nothing to recover from.
Popular in Announcing
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"