@garrison Thanks for your reply! I have rewritten a large part of the code, focusing on making consistency and durability guarantees simpler and better. Main changes:
- MemTable implemented as an ETS table and the store putting SSTs into an ETS table, allowing concurrent readers
- removed all conflict detection between writers and instead implemented one writer at a time and any pending writers are placed in a write queue. This is true serial execution.
- the WAL syncs after every append
Some other changes: CRC verification on SSTs on start, and data in SSTs are compressed for levels larger than 1.
Check the new version out if you have time, any more feedback is welcome and greatly appreciated!
