Mnesia vs Cassandra (vs CouchDB vs ...) - your thoughts?

Seems that Riak Team are working in an implementation called leveled in pure Erlang GitHub - martinsumner/leveled: A pure Erlang Key/Value store - based on a LSM-tree, optimised for HEAD requests · GitHub

This seems to be a strong candidate to be included in the OTP release, what do you think?

Yes, it would. That would be the major difference between the standard Mnesia which uses dets as a backend and one using leveldb. Dets is part of the standard BEAM and implemented in it while leveldb is not. That is one reason I that I expect that leveldb will not be included in the standard OTP release.

As far as I can see, LevelDB itself would need to run outside of OTP, or am I missing something?

(Not that this is a problem per se, but it is something to be aware of.)

It would be nice to see that eleveldb driver that Klarna was working on for Mnesia shipped with OTP, and directly usable. GitHub - klarna/mnesia_eleveldb: An eleveldb backend for Mnesia · GitHub

Do you mean implementing it in the BEAM or implementing it in Erlang in the same way as Mnesia?

I would like to see Mnesia (or something to take its place) be updated to address peoples concerns with using it.

CouchDB is nice but the JSON documents interface is crap, imo.

BEAM could benefit with having a modern distributed DB embedded natively in it ready to go without the little quirks of Mnesia that make it a no go for a lot of use cases.

I realize this is a HUGE task.

Imagine a modern JIT’d BEAM with a blazing fast, native, distributed DB that uses the native Erlang terms that would scale and work right out of the box.

It might not fit for your use case, but ICouch v0.6.2 — Documentation is the most feature complete couch library out there. I don’t use ecto at all yet so I’ve not done any investigation into couch+ecto comparisons.

Riak is maintained now, and is pretty close to a 2.9 release riak/doc/Release 2.9 Series - Overview.md at develop-2.9 · basho/riak · GitHub although I have no idea how you’re supposed to figure that out just by reading the main repo page.

wrt CouchDB’s revision handling yup you are right, however in practice with a front end proxy in front and a sweeper view to handle conflicts I don’t really have issues in practice. It’s possible to get conflicts “inside” a cluster if couchdb can’t meet the requested quorum level (which is more of an admin problem, but …) and accepts a write with 202, and then you manage to write again to the lost node. At least, your data from both writes isn’t lost, and your application can figure out how to merge that as needed.

There’s no particular reason you couldn’t request conflict info on every GET, but personally I prefer 2 other approaches which work for me:

• have a conflicts view that triggers a background task every time it detects a conflict. I have had only a handful of these in years of running CouchDB
• design applications in a conflict-free way such that each write creates a new document, and these can be merged back via a view to only return the most appropriate data

Conflicts are a key part of CouchDB’s mesh architecture if you have multiple clusters, and in practice the issue comes up infrequently.

Given the same N revision docs, couchdb will always pick the same winner (which may not be the one you want) consistently.

Also you can embed your Elixir app “inside” CouchDB as a further supervisor. This is really cool but probably of little practical use for most people.

Depends on what you call “unmaintained”. As far as I know Riak IP was acquired by bet365 last year and Riak has since been moving forward. A new 2.9 RC has been released last month: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2019-January/039316.html
You can see (some?) dev effort here: Commits · basho/riak · GitHub

I guess the sustainability of Riak might be questionable (seems to depend on the will of one company, few developers, etc.). However I came to the same conclusions as you (and partly thanks to these articles) that compared with other solutions it has the best guarantees for eventual consistency (at least for what I plan to do with it).

Note: We are still busy researching the different options here; I am working hard to get Planga up to speed on one of the distributed databases.

So a quick update of my findings w.r.t. CouchDB, Cassandra and Riak in the meantime:

• Cassandra does per-field Last-Write-Wins for conflict resolution.
  • This requires server clocks to be synchronized (!)
  • Also, it is completely unconfigurable.
  • An example: If we have a user structure, and Alice changes user.email and user.phone whereas Bob changes user.phone, assuming Alice’s change happens earlier, the end result will have Alice’s email change and Bob’s phone change in there. (Regardless of if they have seen each-other’s changes in the meantime!)
• CouchDB uses ‘revision hashes’: The revision with the longest history chain wins, with ties solved by taking the revision whose hash is lexicographically higher. The hashes are value-dependent, meaning that if two people perform the same change, there is no conflict.
  • No clocks necessary, CouchDB just uses the observation order of every node separately.
  • However, this means that by default the picked ‘winner’ is essentially random (and might differ between nodes?!), so you have to run your own active conflict-resolution logic.
  • Usually this is done at read-time, by checking if there are conflicts for the resource we want to fetch right now.
• Riak uses either plain get/put, or CRDTs. This means that, as long as you are able to shoehorn your data in the format Riak’s CRDTs use, conflict-resolution is automatic.
  • Riak’s main disadvantage is that it is currently not maintained.

What is a bit unfortunate is that for all three of these systems, the existing database client libraries are all unfinished. It is very likely that we’ll need to write our own Ecto adapter (or, if this turns out to be infeasible, a custom DB wrapper).

Food for thought.

BTW you’re welcome to ask about CouchDB directly - contact me, or the couchdb user mailing list. There’s commercial support available, hosted solutions too, and a BEAM-friendly community as well on IRC. See the https://couchdb.org/ website for more info.