Unexpected behaviour with mnesia - not actually durable?

Like most other dbases Mnesia’s disc_only_copies / disc_copies writes to a write-ahead-log. The abrupt termination didn’t allow it to sync to disc properly causing the error you see.

Unfortunately there is not really much you can do to avoid this situation. Even the “robust” Oracle ended up with a very corrupted dbase after dripping water from condensation fried the server.

In general a kill -15 should be attempted first to allow the application a chance to cleanup before a kill -9.

While I don’t recall to see any warning is it possible that a not proper closed table can cause the lost of all records on it?

I have this happen to me after I have forced the running application to terminate with ctrl + c + c:

Extracted from Mnesia And The Art of Remembering | Learn You Some Erlang for Great Good!

Screenshot from 2020-12-18 10-11-55682×182 32.7 KB

Maybe @ferd can shed us some light why this isn’t the case of being 100% guaranteed in your use case?

So this lost of data when you kill Mnesia is not due at all to Mnesia itself, but instead to how the file is open to write to the disk,

Please see this post that also as a script demonstrating the same issue when writing directly to the disk from a script:

After originally posting here, I dove into the Erlang source code and posted on the Erlang bug tracker: Redirecting…

I’m pretty far removed from this issue at the moment, so may not have much further context to share, but I believe between this thread and the one in the bug tracker, everything I observed has been captured, in case it’s useful to anyone. Looking at the Erlang source code is likely provide a lot of clarity around where any blocking actually occurs, and echoing what has been said by a few folks in a few different places, it isn’t always what you might assume or expect in regards to durability, based on the documentation.

The issue It’s not related with any blocking in Erlang per se… As I mentioned in the post I linked previously, it’s because the BEAM has a default of 2 seconds to write data to the disk when the file is open with the defaults for delayed_write, as per docs :

delayed_write

The same as {delayed_write, Size, Delay} with reasonable default values for Size and Delay (roughly some 64 KB, 2 seconds).

You can confirm this with a script that writes directly to the file system:

defmodule FileIO do
  
  @moduledoc """
  iex> fd = FileIO.open! "test.txt"                             
  #PID<0.292.0>
  iex> FileIO.append!(fd, "test it") && FileIO.read!("test.txt")
  "test it\n"
  iex> FileIO.close fd                                          
  :ok
  """

  @write_mode [:append, :binary]
  # @write_mode [:append, :binary, :delayed_write]
  # @write_mode [:append, :binary, {:delayed_write, 1, 1}]

  def open!(path, write_mode \\ @write_mode) do
    File.open!(path, write_mode)
  end

  def close(file_descriptor) do
    File.close(file_descriptor)
  end

  def append!(file_descriptor, data) do
    {:ok, start_position} = :file.position(file_descriptor, :cur)

    :ok = IO.binwrite(file_descriptor, "#{data}\n")

    {:ok, end_position} = :file.position(file_descriptor, :cur)

    %{
      file_descriptor: file_descriptor,
      start_position: start_position,
      end_position: end_position,
      size: byte_size(data)
    }
  end

  def read!(path) do
    {:ok, data} = :file.read_file(path)
    data
  end
end

So, I can confirm that the delay of 2 seconds is indeed coming from the BEAM when the file is open to write with delayed_write. If in the above script I use @write_mode [:append, :binary] or @write_mode [:append, :binary, {:delayed_write, 1, 1}] I can immediately read the content of the file after I write to it and see that my last write is persisted on disk, but if I use instead @write_mode [:append, :binary, :delayed_write] I cannot see the last write in the file, unless I wait 2 seconds to read it.

I totally agree with you here…

In my opinion calling Mnesia ACID, therefore durable is stretching a little the definition of ACID and durable, but it seems the reply is always replication, that then leads to netsplits, that then leads to data loss when recovering from it, thus the ACID definition is again being streched.