I know we (one of my co-workers) tried a proof of concept with Mnesia and were unable to get the performance we needed on our current project – this could’ve been due to problems with our implementation or maybe it didn’t quite match our needs, I’ll have to ask to find out more.
Write performance in distributed mnesia is a bit limited and depends on the number of nodes and transaction type (async_dirty/sync_dirty/async_transaction/sync_transaction). I also remember something about a bottle-neck in file handling as there was only 1 process that wrote everything meaning you can’t split out different tables to different physical disk controllers to increase throughput (this was one thing that whatsapp patched in mnesia)
On moderate hardware we maxed out somewhere around 3000 writes/second using sync_transaction and 3 nodes. It is prone to overload and it wont stop you from doing it . It helps if you limit concurrency to the number of schedulers.
Do you have a link to any papers/documentation where they explain what algorithms and technology they use? Their homepage was too high level and didn’t contain much information and wikipedia is light on details as well.
I’d very highly recommend watching https://www.youtube.com/watch?v=4fFDFbi3toc. They were running 10 million scenarios per night to test the thing when they were a startup I can only imagine what they were able to do with Apple’s resources. Given the approach they use it’s extremely likely it actually is the most reliable and tested distributed KV store.
it’s an ordered KV store. You can use it as a storage layer for RDBMS and my understanding is that there is SQL RDBMS that uses foundationDB but it was not open sourced by Apple. I bet FoundationDB will get picked up by various projects as underlying storage engine so hopefully we will see in not too distant future a number of SQL RDBMS using FoundationDB as a storage layer. Would be cool if it was PostgreSQL but it’s unlikely because it doesn’t have pluggable storage engines.
Yep but it’s ACID and explicitly designed to be a storage layer for a higher level database. They wrote toy implementation of SQLite on top of FoundationDB in a day. Plus again thanx to the way they have developed the thing it’s most likely the most reliable distributed KV store.