Hi @tduccuong , thanks for the kind words

My question is whether this approach makes sense for CubDB?

I don’t see problems with your approach, but it could be simpler. If you use a RAM disk, you probably do not need complete durability in case of a restart, and you are ok loosing very recent logs (after the most recent disk copy). In that case, you could do without a RAM disk, and simply use the option auto_file_sync: false. This will avoid flushing buffers to disk for each transaction, and instead let the OS control the flush to disk. You can also explicitly call CubDb.file_sync(pid) periodically to force a flush. In all likelihood this would cause disk writes more frequently than every 2 hours though, so if that’s a problem you might still follow your approach.

You can check if there is an ongoing compaction by calling CubDB.compacting?(pid), or you can disable auto compaction completely with the auto_compact: false option, and force a compaction manually when you see fit, with CubDB.compact(pid). The latter option can be especially useful if you have “quiet times” when you can perform a large compaction, and “busy times” when you prefer not to add overhead. Delaying a compaction won’t affect read or write performance, and will simply cause the database file to be larger than it could. Compactions are also more efficient during quiet times, when the database does not need to catch up on writes that happen after compaction is started.

I hope this helps

Never mind my last question, I got the answer after going through your code base. I think my approach now is:

• auto_file_sync = false
• Build a BTree in memory as buffer, that exposes same API as CubDB (put/2, put_multi/2) for client code
• Periodically flush the BTree to disk via CubDB.put_multi/2

Do you think this helps protect the SD card better?

Thanks for your help!

Consider that CubDB is already structured as an append-only btree, so at the file level writes are always appended (even in case of an update or a deletion). Therefore, the write characteristics are very similar to a plain log file, but with the added ability to perform point and range queries efficiently.

What’s the expected write volume in terms of entries per unit of time? Depending on it, it might be worth using CubDB straight away, with the options discussed above.