Suppose we are on linux x86_64.
Suppose, in the middle of a file system write / fsync, someone pulls the power and the machine shuts down.
When the machine boots up, because we were in the middle of a partial write, there is not all that much guarantees on the state of the file system.
Question: Does Erlang/Elixir have any libraries / data structures (besides ‘use a database engine’) for handling situations like this?
So again, the problem here, is not a single process crashing; the problem here is that the entire machine loses power in the middle of a write / fsync, the file being written to is in some inconsistent state, and we are trying to recover from this.
There is disk_log which mentions in the docs:
Supports automatic repair of log files that are not properly closed…
Probably using sqlite instead of the naked FS will solve most of the potential problems. It has a lot of guards against corrupted state.
This is my fault for not clearly stating this in the question: I’m playing with Write-Ahead-Logs, Log-Structured-Merge trees, append-only-b-trees, and a few other data structures.
So yes, sqlite would solve this problem, but it’s not what I want because sqlite commits to a particular on “can recover from crash disk representation”. (last I checked, it defaults to write-ahead-log).
I’m interested in exploring the space of other “can recover from crash disk representation”
Always been interested in the same but life and work keep happening so never explored it in more detail.
Sorry that I can’t help further.