Is there a pure Elixir KV Store?

Check out ETS, here’s the elixir school page Erlang Term Storage (ETS) · Elixir School

Technically speaking ets isn’t equivalent to BoltDB since ets is not persistent. For that you’d have to look at dets or mnesia. If you’re just looking for LMDB then there’s a good nif for it here: GitHub - zambal/elmdb: Erlang NIF driver for LMDB, a fast, compact and transactional key-value data store · GitHub.

While GitHub - LMDB/lmdb: Read-only mirror of official repo on openldap.org. Issues and pull requests here are ignored. Use OpenLDAP ITS for issues. · GitHub seems to be alive the Erlang NIF seems to be dead

Any real world example of projects in Elixir/Erlang using LMDB in production?

I don’t know of any. I also don’t know that I’d argue that the nif is dead but you’ll have to make that judgement for yourself. As far as other KV stores go I’ve used rocksdb | Hex a ton and really like it. If what you need is “KV Store” and not “LMDB” (which is what boltdb is based on) then I can highly recommend that rocksdb nif.

I am just exploring my way into Elixir world and a lot curious about everything not really on the need of anything in special for now.

Why not Mnesia instead of other KV stores?

PS: nice podcasts

Thanks for the links. Are there any performance benefits to having a native Elixir implementation vs an NIF? Similar to what Golang has with BoltDB?

Preference I guess. Mnesia provides more features then either rocksdb or lmdb but also has some issues with file loading times, maximum file sizes, repair times on corrupted files, etc. IIRC Klarna built a backend for mnesia in leveldb (which is what rocksdb is based on) in order to solve those issues. If you just need keys and values on a single file system then rocksdb is what I would use. If you want the extra features of mnesia and you don’t have a ton of data to store then mnesia is probably fine.

Aw thanks! Happy to hear you’re enjoying it.

If you just need a quick lookup for KVs and don’t need persistence then ETS is going to be your best bet. I haven’t benchmarked dets or any other “native” elixir solution vs. the rocksdb nif so I can’t say what the performance implications are. Personally I use rocksdb because its “very fast” and its seen tons of use across many companies and solutions. So I trust it to be reliable.

We do need persistence. I should of been more specific with the topic title. RocksDB and it’s predecessor LevelDB are fine for smaller database applications but as the database grows into the double digit GB range and above, the performance degradation tends to be quite noticeable.

Well DETS (which is what mnesia is based on) has a max 2gb file size so you’ll have to use mnesia’s table fragmentation anyway in that case.

I’m assuming you mean read performance here. I’m also assuming you’ve tuned your caches, compaction threads and such for this? In my experience you shouldn’t have that much issue tuning reads in the “double digit GB” order of magnitude but obviously you know your use case more then me . I’m also not arguing that you wouldn’t get better read performance from LMDB or any other db based on B+trees .

All of that aside, I don’t know of a KV store written in Elixir that I would trust that hasn’t already been mentioned or isn’t based on an underlying DB engine through NIFs.

For local KV storage I tend to use either DETS (single-load use), mnesia (persistant save/load use), or leveldb (for high speed data that might exceed 2 gigs).

Overall I default to mnesia.