-
Suppose we are on linux x86_64.
-
Suppose, in the middle of a file system write / fsync, someone pulls the power and the machine shuts down.
-
When the machine boots up, because we were in the middle of a partial write, there is not all that much guarantees on the state of the file system.
-
Question: Does Erlang/Elixir have any libraries / data structures (besides ‘use a database engine’) for handling situations like this?
So again, the problem here, is not a single process crashing; the problem here is that the entire machine loses power in the middle of a write / fsync, the file being written to is in some inconsistent state, and we are trying to recover from this.
There is disk_log which mentions in the docs:
internal format
Supports automatic repair of log files that are not properly closed…
1 Like
Probably using sqlite instead of the naked FS will solve most of the potential problems. It has a lot of guards against corrupted state.
This is my fault for not clearly stating this in the question: I’m playing with Write-Ahead-Logs, Log-Structured-Merge trees, append-only-b-trees, and a few other data structures.
So yes, sqlite would solve this problem, but it’s not what I want because sqlite commits to a particular on “can recover from crash disk representation”. (last I checked, it defaults to write-ahead-log).
I’m interested in exploring the space of other “can recover from crash disk representation”
1 Like
Always been interested in the same but life and work keep happening so never explored it in more detail.
Sorry that I can’t help further.