FoundationDB by Apple; a great match for Elixir?

databases
foundationdb
Tags: #<Tag:0x00007f039bd62f90> #<Tag:0x00007f039bd62e00>

#1

I was excited to see the open sourcing of FoundationDB by Apple; it looks like it would be a great match for Elixir.

Ruby already has bindings for it – maybe someone could translate those into an Elixir adapter?

Discussion on how to go about it:

Example bindings:

-Scott S.


#2

What makes FoundationDB stand out?


#3

Probably the distributed nature of FoundationDB…


#4

What about Mnesia :smiley: written in Erlang :003:

Good thread on distributed DBs here too:


#5

Mostly the distributed fault-tolerance with ACID transactions.

Many dev comments on it here:
https://news.ycombinator.com/item?id=16877401


#6

I know we (one of my co-workers) tried a proof of concept with Mnesia and were unable to get the performance we needed on our current project – this could’ve been due to problems with our implementation or maybe it didn’t quite match our needs, I’ll have to ask to find out more.


#7

Write performance in distributed mnesia is a bit limited and depends on the number of nodes and transaction type (async_dirty/sync_dirty/async_transaction/sync_transaction). I also remember something about a bottle-neck in file handling as there was only 1 process that wrote everything meaning you can’t split out different tables to different physical disk controllers to increase throughput (this was one thing that whatsapp patched in mnesia)

On moderate hardware we maxed out somewhere around 3000 writes/second using sync_transaction and 3 nodes. It is prone to overload and it wont stop you from doing it :smiley: . It helps if you limit concurrency to the number of schedulers.

Reads are usually faster than any other solution.


#8

due to the way it was/is developed it’s most likely the most reliable distributed kv database out there.


#9

That is a pretty big claim :smiley:

Do you have a link to any papers/documentation where they explain what algorithms and technology they use? Their homepage was too high level and didn’t contain much information and wikipedia is light on details as well.


#10

There is a bit more information on the forum
https://forums.foundationdb.org/t/technical-overview-of-the-database/135 and in the comment section of the announcement https://news.ycombinator.com/item?id=16877395.


#11

This video by one of the authors about how they were testing it is worth a watch https://www.youtube.com/watch?v=4fFDFbi3toc


#12

I’d very highly recommend watching
https://www.youtube.com/watch?v=4fFDFbi3toc. They were running 10 million scenarios per night to test the thing when they were a startup I can only imagine what they were able to do with Apple’s resources. Given the approach they use it’s extremely likely it actually is the most reliable and tested distributed KV store.


#13

how does it compare to riak?


#14

I think the biggest difference is that FoundationDB supports ACID with linearizable consistency. It also seems to have some limited relational support.


#15

?
it’s an ordered KV store. You can use it as a storage layer for RDBMS and my understanding is that there is SQL RDBMS that uses foundationDB but it was not open sourced by Apple. I bet FoundationDB will get picked up by various projects as underlying storage engine so hopefully we will see in not too distant future a number of SQL RDBMS using FoundationDB as a storage layer. Would be cool if it was PostgreSQL but it’s unlikely because it doesn’t have pluggable storage engines.


#16

Hmm, how does it compare to leveldb?


#17

It’s distributed


#18

So it’s more like Riak then? Riak uses leveldb to distribute commands amongst a quarom of nodes that can be dynamically added/removed.


#19

Yep but it’s ACID and explicitly designed to be a storage layer for a higher level database. They wrote toy implementation of SQLite on top of FoundationDB in a day. Plus again thanx to the way they have developed the thing it’s most likely the most reliable distributed KV store.


#20

That sound very slow, how do they pull that off?