yoavgeva

yoavgeva

FerricStore - a durable, Redis-compatible cache

Hey everyone,

I’ve been working on something I want to share and get feedback on.

The itch

Every web app I build ends up with the same stack: a database, a cache layer, and the app itself. Three things to deploy, monitor, pay for, and hope they stay connected to each other.

For caching, the options are:

Redis — great protocol, great ecosystem, but in production you need a managed service (ElastiCache, Upstash, etc.) because self-hosted Redis persistence is… optimistic. AOF rewrite can lose your last few seconds. RDB snapshots lose everything since the last dump. And now you’re paying for a whole separate instance just to hold keys.

Java solutions* (Hazelcast, Infinispan) — powerful but painful to operate. Config files longer than your app code. JVM tuning. Cluster formation issues. Not exactly “easy to maintain.”

ETS/Mnesia — built into the BEAM, fast, but volatile. Node goes down, data’s gone. Mnesia replication exists but comes with its own set of surprises.

What I actually wanted: use the disk I’m already paying for on my instance, speak a protocol I already know (Redis), get real durability without a separate service, and keep my stack simple.

What I built with AI

[FerricStore]( GitHub - yoavgeva/ferricstore: Distributed crash-safe key-value store with Redis wire protocol. Durable by default — every write is Raft-committed and fsync'd. Embeddable Elixir library or standalone server. Built with Elixir + Rust NIFs. · GitHub ) is a persistent key-value store written in Elixir + Rust that:

Speaks RESP3 — connect with redis-cli, or any Redis client library that support RESP3. Your existing code mostly just works.

Every write is durable by default — Raft consensus + Bitcask append-only log + fsync. When you get OK back, your data is on disk. Not “eventually.” Not “if the AOF rewrite finishes.” On disk, right now.

Runs embedded in your app — add it as a dependency, FerricStore.set("key", "value"). No separate process, no network hop, no connection pool. Your cache lives in the same BEAM node as your app.

Or runs standalone — `docker run -p 6379:6379 yoavgeva/ferricstore` and connect with redis-cli. Drop-in for apps that already speak Redis.

50+ Redis commands — strings, hashes, lists, sets, sorted sets, TTL, MULTI/EXEC, pub/sub, and more.

Native Elixir commands — CAS (compare-and-swap), distributed locks, rate limiting, FETCH_OR_COMPUTE — things you’d build on top of Redis but get out of the box here.

Probabilistic data structures — Bloom filters, Cuckoo filters, Count-Min Sketch, TopK, HyperLogLog, T-Digest. All built-in.

Vector search — HNSW index for similarity search, built into the storage engine.

The architecture in 30 seconds

Client (redis-cli / Redix / your app)

|
v

RESP3 Parser (pure Elixir, zero-copy)

|
v

Command Dispatcher

|
v

Raft Consensus (via ra library — same one RabbitMQ uses)

|
v

Bitcask Storage Engine (Rust NIF — append-only log, CRC-checked)

|
v

ETS Hot Cache (recent values served from memory, cold values read from disk)

Writes go through Raft for consistency, then to Bitcask for persistence. Reads hit ETS first (microseconds), fall back to disk for cold data. The Rust NIF handles the low-level I/O — pure functions, no dirty schedulers, proper consume_timeslice yielding so the BEAM scheduler stays happy.

Embedded mode — the thing I’m most excited about



# mix.exs

{:ferricstore, "\~> 0.1"}



# your app

FerricStore.set("user:123:session", session_data)

{:ok, data} = FerricStore.get("user:123:session")



# with TTL

FerricStore.set("rate:api:123", "1", ttl: :timer.seconds(60))



# atomic operations

{:ok, new_count} = FerricStore.incr("page_views")



# compare-and-swap

:ok = FerricStore.cas("inventory:sku42", "10", "9")


No Redis connection. No connection pool config. No “what happens when Redis is down.” It’s just a function call that persists to disk. Your Phoenix app, your cache, one deployment, one thing to monitor.

Standalone mode — drop-in Redis replacement

If you have apps in other languages, or you just want a Redis-compatible server with real durability:

bash

# Docker

docker run -p 6379:6379 -v ferricstore_data:/data yoavgeva/ferricstore



# Then use any Redis client

redis-cli SET mykey "hello"

redis-cli GET mykey


Or from your Elixir app via Redix:

elixir

{:ok, conn} = Redix.start_link("redis://localhost:6379")

Redix.command!(conn, \["SET", "user:42", "alice"\])

Redix.command!(conn, \["GET", "user:42"\])

# => "Alice"


Everything you’d expect from Redis works — MULTI/EXEC transactions, pub/sub, pipelining, HELLO 3 (RESP3). Plus you get a built-in health endpoint (GET /health on port 6380), Prometheus metrics, ACL authentication, and TLS support.

The dashboard gives you live visibility into shards, key counts, memory pressure, hit rates, and slow queries — no Grafana setup needed.

What it’s NOT

Not a database replacement — it’s a cache/store. Great for sessions, rate limits, feature flags, job queues, counters, leaderboards. Not for your users table.

Not a sharded cluster (yet) — scales to 3-5 nodes via Raft replication for high availability and read scaling (every node serves reads from local ETS). Storage capacity scales with disk, not RAM — a 500GB NVMe gives you 500GB of cache. But there’s no hash-slot sharding across nodes yet, so every node holds all the data.

Not battle-tested in production — this is v0.1. It passes 8000+ tests including shard-kill recovery and multi-node cluster tests, but it hasn’t seen real production traffic yet. That’s where you come in.

Not going to beat Redis on raw throughput — Redis keeps everything in RAM and doesn’t fsync by default. But it’s persistent store where every write hits disk,

Why Elixir + Rust?

Elixir gives us the BEAM’s supervision trees, distribution primitives, and ETS for the hot cache. Rust gives us a memory-safe, zero-copy storage engine that doesn’t GC-pause or mess with the BEAM scheduler.

The NIF boundary is clean — Rust functions are pure and stateless. No Mutex, no shared state, no dirty schedulers. Just `v2_append_batch(path, entries)` and `v2_pread_at(path, offset)`.

Looking for

Feedback on the API, the architecture, the docs — what’s confusing, what’s missing, what would make you try it?

Early adopters willing to try it in a side project or staging environment and report what breaks.

Use case ideas — what would you use a durable, embedded Redis-compatible cache for in your Elixir apps?

Links:

- Hex: ferricstore | Hex

- Docs: ferricstore v0.5.7 — Documentation

- Docker: yoavgeva/ferricstore - Docker Image

- GitHub:

https://github.com/yoavgeva/ferricstore

Happy to answer any questions. And yes, the name is a pun — Ferric (iron/Rust) + Store.

Most Liked

AstonJ

AstonJ

A few days ago we updated the flag system after having changed the post composition tip to the following a few weeks ago:

Type here. Use Markdown, BBCode, or HTML to format. Drag or paste images. Please use your own words (no AI slop please).

The flag system:

It’s AI Generated

Using AI as a smart dictionary, thesaurus, or translator is ok - using AI for content composition (or posting anything that looks like it) is not.

We’ve been monitoring AI generated posts in a log since Feb of last year and determined this is the best way forward for now.

If people want to post AI generated text/conversations, then the new section we created last year may be a good fit as it would ensure the flow of the main thread remains in line with expectations/the norm: https://forum.elixirforum.com/t/new-ai-conversations-section/72455 (we could probably update those guidelines now).


To the OP, you should be able to edit the post and submit a new version (just please make sure it’s in your own words - translated to English with the help of an AI is fine if need be).

Asd

Asd

Hi, good library,

It’s a very big project (30k lines of Elixir code, 15k lines of Rust and around 100k lines of tests) implemented in only two weeks according to git history.

I am reading the code and it is… strange.

First of all, it is a cache, but it has strange eviction logic. If write too many keys (95% of keydir memory limit), the FerricStore will enter a keydir_pressure_level = :reject state, where it will fail all incoming set calls. Entering this state also triggers the eviction, but it will only be triggered asynchronously. And even after this eviction happens, I won’t be able to write new keys until the MemoryGuard process performs another check (that is for 100ms). I mean, it would make sense to block the writes, but it appears to just return :error and not pass any set further.

Then, its RAFT, which is strange given that your post says “no network hop”. The problem is that async writes do not check for leadership change. In the end, async means that cluster can be in an inconsistent state and one node may see different results from the other. Let me explain this, for example, node A performs async set("x", 1) and node B performs async set("x", 2). Async write is implemented in a way that you first write to ETS, then disk storage, then you fire-and-forget into RAFT cluster. But this fire-and-forget operation may not succeed (because leader changed or any other issue, its consensus after all). So, if this issue happens with the second write, you are going to end up in a situation where node A sees "x" = 1 and node B sees "x" = 2. That may be fine, but the worst thing is that this situation is never fixed. It makes this async_write not even eventually consistent. It is just inconsistent, forever. I’d expect async writes to be eventually completed if the system is alive.

Then, every time one shard log file collects more than 256MB of data (that value is hardcoded), it will do a persistent_term.put, triggering a global GC. Not a big deal, but a strange decision.

Then, Bitcask compaction is triggered in a scheduled way. It attempts to compact every 30 seconds, and at that time it performs multiple File.ls calls. I think that it is possible to check if compaction needs to be triggered when rotating the shard log file, when performing a check.

And yeah, whole idea to use bitcask is very strange. How bitcask works: it just appends new entries into file and then puts them in a big hash table like key -> log_file_name_and_offset. That’s it. You use an ETS as a hot cache alongside a bitcask. That is strange. Why not just use an append-only file (or files) and a one or two ETS tables (or :shards) which stores key -> value or log_file_name_and_offset. That’s minus 15k lines of Rust, no NIF required. In the end, its an IO-bound task and there’s no need to use NIF in the first place. If you did NIF for performance, I’d suggest optimizing the code and reviewing the performance manually. I suspect that Claude performed some flamegraph analysis on hot paths and optimized code for them (thats obvious with comments like “I do … in order to save 10 nanoseconds”, that made me laugh :grin:), but that resulted in benchmark cheating, where all expensive operations were moved to the background (like compaction), and were left without optimization.

Then, Sandbox code is spread around all the project, introducing lots of ifs in all the modules. I’d suggest to just start separate instances of the FerricStore in tests, one for each test. That would make the code much more simple.


In the end, congratulations on initial release! :tada: I don’t think that I will be using this project in its current state. I’d also suggest to not feed this review into Claude, because I think it would be much better if you used this review as a starting point in your own review of the code in order to learn more about database and caches. I can perform a more detailed review if you want (for a reasonable price of course).


I’d suggest checking out marvelous Nebulex. It has no built-in persistence, but you can use any storage as an adapter (or implement your own storage).

dimitarvp

dimitarvp

Nice! Embedded is an actual selling point for me, though at that point you’d be competing with ETS.

I also expect @Asd to come and review your library top to bottom any moment now. :smile:

Where Next?

Popular in Announcing Top

martinthenth
Hello everybody :wave: Recently, some of my colleagues talked about database ids and uuids and their problems, and I remembered the pain...
New
pkrawat1
Presenting Aviacommerce, open source e-commerce platform in Elixir Aviacommerce is an open source e-commerce platform in Elixir. We at...
New
ostinelli
Let’s write a database! Well not really, but I think it’s a little sad that there doesn’t seem to be a simple in-memory distributed KV da...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
Crowdhailer
Experimenting with this code. OK.try do user <- fetch_user(1) cart <- fetch_cart(1) order = checkout(cart, user) save_orde...
New
Qqwy
Hello everyone, I wrote a small library today called MapDiff. It returns a map listing the (smallest amount of) changes to get from map...
New
mindok
What is ContEx? A pure Elixir server-side data plotting/charting library outputting SVG. It has nice barcharts in particular and works g...
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New
trisolaran
Hi! :waving_hand: I would like to present LiveSelect, a little library that I wrote to easily add a dynamic selection input to your LV f...
198 10858 107
New
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement