Qqwy

Qqwy

TypeCheck Core Team

Mnesia vs Cassandra (vs CouchDB vs ...) - your thoughts?

Looking at the stacks that existing large companies have used, WhatsApp internally uses Mnesia to store the messages, while Discord uses Cassandra.

Also, in the past we’ve had quite a long discussion about ‘When to use Mnesia’ (vs. a traditional relational database), but this was about a year ago, and maybe there have been changes to the Mnesia Ecto bindings or similar?


What I already know:

  1. Cassandra runs on Java, Mnesia runs inside your BEAM.
  2. Both are distributed databases.
  3. Mnesia does not do conflict resolution (although you can write your own, and the packages unsplit and [reunion](GitHub - snar/reunion: Mnesia partition handler · GitHub exist to do this for you). Cassandra always uses last-write-wins for conflict resolution.

Most Liked

rvirding

rvirding

Creator of Erlang

No, mnesia has always had the ability to store data on disk. You just specify on which nodes you want disc copies. The only earlier limitation has been that the disk files must not be larger than 2 Gb but you got around this by partitioning your tables.

OvermindDL1

OvermindDL1

Going with the last few posts, put it all behind a module interface, start with the most basic, which might just be ETS or so in RAM, maybe mnesia later, maybe move to eleveldb or something later, maybe even migrate to PostgreSQL later, all without changing the interface.

dch

dch

I saw your request on the couchdb mailing list, and while it’s “not supported” it’s not very hard to do it either, as this is actually how CouchDB is built internally anyway.

These notes are for 1.7.x which is what I’ve got to hand:

  • create your elixir application as usual and compile it
  • unpack the release into couchdb plugins dir (or /usr/local/lib/couchdb/erlang/lib/... ) for whatever your OS uses
  • add a [daemons] section to your local.ini config with myapp={'Elixir.MyApp', start_link, []} which will ensure it gets started up as part of the main supervision tree of couch
  • set ERL_FLAGS environment variable to use -config /usr/local/etc/couchdb/sys.config to get your custom application settings loaded up
  • look through https://github.com/apache/couchdb/tree/master/src/couch_plugins which has some useful comments and source snippets

I’m reasonably sure this is complete, and while the complexities of 2.x series with cluster support make this moire complicated, the guts of that should remain the same.

At some point, somebody will decide to put a nice tidy shim in between the chttpd “clustered httpd” layer that exposes the soft erlang underbelly of couchdb, and then you’d have native erlang terms accessible to Elixir as well. There are a few not-so-handwavey details to deal with - no maps yet as couch uses proplists (no maps at the time in Erlang, and to support some quirks of JSON), things like structs have no way of being converted back to JSON at the http layer, and a bunch of valid BEAM types that would need to be cleaned when turning them into JSON again etc etc, once the shim is available.

The big issue is that releases for a clustered db are a serious thing, and we tend to update and restart our apps far more often than our DBs. If you’re already co-locating the DB and the app on the same server, only the JSON encoding & recoding is the bottleneck.

Where Next?

Popular in Discussions Top

Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
cvkmohan
The upcoming Phoenix 1.6 release looks very interesting. Became a habit to watch the commits - and - what they are bringing in. phx.gen...
New
mmport80
I have put far too much effort into Dialyzer over the last year or so - and basically - I doubt it’s worth the effort. It’s not as easy ...
New
fireproofsocks
This is more of a general question, but I’m wondering how other people in the community think about the pattern matching in function sign...
New
Ankhers
Just a little information upfront. Generally speaking, if I feel like I need to either break a pipe chain or use an anonymous function in...
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
tomekowal
Hey guys! I want to create a toy project that shows a chart of temperature over time and updates every 5 seconds. I feel LiveView is per...
New
acrolink
How does the two languages compare when it comes to server side application development? Any experiences or ideas? Thank you.
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
kostonstyle
Hi all How can I compare haskell with elixir, included tools, webservices, ect. Thanks
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

We're in Beta

About us Mission Statement