To answer more generally, from a theory perspective an event log and a “database” are equivalent as discussed earlier.

But in practice performance matters very much. Storing all of your events in an immutable log (i.e. event sourcing in the limit) means that reading state is extremely expensive. It also means that nothing is ever deleted, which is a problem in practice.

There are “log-structured” storage engines which are designed to soften the blow, like an LSM tree. LSMs make tradeoffs where they keep keys sorted (for faster binary search) but they are split into separate “files” which are immutable and then merged later. The merge also ensures old entries are (eventually) deleted. But once you’re using an LSM you have already departed the “log” abstraction for a K/V abstraction, the storage is just vaguely log-structured.

But the thing is, order of keys matters a lot for many use-cases too. The log-structured runs all have to be scanned (and sometimes they overlap), so range queries (and especially range deletes) are expensive. For point lookups you can use bloom filters, but for range lookups it is not so simple.

So what I’m getting at is you want a distributed log/wal for correctness because it makes replication easier, but you want your actual data materialized into a format which is useful for lookups like an LSM or, even better, a btree. That is exactly what FoundationDB does.

The WAL is used for replication, and crucially it can be truncated during recovery if a node fails while a transaction is only partially replicated. Then the storage engines can pull data from the log and put them into an ordered (btree) format for performant queries.

The advantage of this log approach becomes more clear the deeper you get. For example, how do you “move” a shard of data from one set of servers to another while serving fully consistent reads and writes without any unavailability?

If you already have a 100% consistent and durable log (the hard part), it’s actually rather simple: you pull the keys in chunks from one storage engine to the other and then replay the log over the new shard as you go. If the log operations are idempotent (true for simple sets/clears) then this is correct. What’s critical about this algorithm is that nothing is blocked for very long, as it can all be done incrementally. And since everything is properly versioned by the transaction system, an inconsistent snapshot is never observed by any client.

It goes deeper, though! How does a client know to read from the old shard until exactly the moment that the new shard is ready, and then switch over? Any inconsistency here would break the correctness guarantees. Well, it turns out it’s (relatively) simple: the “changeover” is part of the database, and therefore part of the log.

FDB really is a beautiful database.