HammerActually

HammerActually

Unexpected behaviour with mnesia - not actually durable?

I’m observing behavior with mnesia that is unexpected by me and could use some help understanding what’s going on. I’ve tested this with erlang 21.0.1 and 21.2.2, and elixir 1.7.4.

Given two scripts:

write.exs

IO.puts "Creating schema..."
:ok = :mnesia.create_schema([node()])
IO.puts "Starting mnesia..."
:ok = :mnesia.start()
IO.puts "Creating table..."
{:atomic, :ok} = :mnesia.create_table(:example, [{:disc_only_copies, [node()]}, type: :set, attributes: [:name, :value]])
IO.puts "Setting :foo = 1..."
{:atomic, :ok} = :mnesia.transaction(fn -> :mnesia.write({:example, :foo, 1}) end)
IO.puts "Looking up :foo..."
:mnesia.transaction(fn -> :mnesia.read({:example, :foo}) end) |> IO.inspect
IO.puts "Done."

read.exs

IO.puts "Starting mnesia..."
:ok = :mnesia.start()
IO.puts "Waiting until table is loaded..."
:mnesia.wait_for_tables([:example], 1_000)
IO.puts "Looking up :foo..."
:mnesia.transaction(fn -> :mnesia.read({:example, :foo}) end) |> IO.inspect
IO.puts "Done."

When I run these scripts:

rm -rf Mnesia.nonode@nohost; mix run write.exs; mix run read.exs

I observe the following:

Creating schema...
Starting mnesia...
Creating table...
Setting :foo = 1...
Looking up :foo...
{:atomic, [{:example, :foo, 1}]}
Done.
Starting mnesia...
Waiting until table is loaded...
dets: file "Mnesia.nonode@nohost/example.DAT" not properly closed, repairing ...
Looking up :foo...
{:atomic, []}
Done.

The primary behavior that is unexpected to me is that writing to a disc only table inside a transaction doesn’t seem to ensure that the record is actually written to disc before moving on after the transaction. ie, it’s not actually durable. I do observe the dets warning that the table file was not properly closed, which I presume is because the script terminated abruptly (though normally).

If I explicitly call :mnesia.stop before the end of the write.exs script, the record is actually persisted and successfully read by the read.exs script, however I expect that I should be able to trust that once the transaction returns, the record is safely persisted. I can include :mnesia.stop in my shutdown procedures, but it does not appear that it’s actually durable in the case where the application crashes or is halted via kill -9.

If instead of stopping :mnesia I sleep for a few seconds at the end of write.exs, the dets warning is still emitted but the record is successfully found. So it does eventually write the record to disc without an explicit :mnesia.stop, but just not before the :mnesia.transaction call returns, which very clearly to me seems to conflict with what is said here: Transactions and Other Access Contexts — OTP 29.0.2 (mnesia 4.26.1)

I’ve observed the same behavior with :disc_copies and using :mnesia.sync_transactionas well, just to be thorough. The following is the output from:mnesia.system_info(:all)`, in case it’s useful. It’s all default values afaik. (paths have been truncated by me)

[
  access_module: :mnesia,
  auto_repair: true,
  backend_types: [:ram_copies, :disc_copies, :disc_only_copies],
  backup_module: :mnesia_backup,
  checkpoints: [],
  db_nodes: [:nonode@nohost],
  debug: :none,
  directory: 'Mnesia.nonode@nohost',
  dump_log_load_regulation: false,
  dump_log_time_threshold: 180000,
  dump_log_update_in_place: true,
  dump_log_write_threshold: 1000,
  event_module: :mnesia_event,
  extra_db_nodes: [],
  fallback_activated: false,
  held_locks: [],
  ignore_fallback_at_startup: false,
  fallback_error_function: {:mnesia, :lkill},
  is_running: :yes,
  local_tables: [:schema, :example],
  lock_queue: [],
  log_version: '4.3',
  master_node_tables: [],
  max_wait_for_decision: :infinity,
  protocol_version: {8, 3},
  running_db_nodes: [:nonode@nohost],
  schema_location: :opt_disc,
  schema_version: {2, 0},
  subscribers: [#PID<0.141.0>],
  tables: [:schema, :example],
  transaction_commits: 2,
  transaction_failures: 0,
  transaction_log_writes: 0,
  transaction_restarts: 0,
  transactions: [],
  use_dir: true,
  core_dir: false,
  no_table_loaders: 2,
  dc_dump_limit: 4,
  send_compressed: 0,
  version: '4.15.5'
]

Most Liked

tty

tty

Like most other dbases Mnesia’s disc_only_copies / disc_copies writes to a write-ahead-log. The abrupt termination didn’t allow it to sync to disc properly causing the error you see.

Unfortunately there is not really much you can do to avoid this situation. Even the “robust” Oracle ended up with a very corrupted dbase after dripping water from condensation fried the server.

In general a kill -15 should be attempted first to allow the application a chance to cleanup before a kill -9.

HammerActually

HammerActually

After originally posting here, I dove into the Erlang source code and posted on the Erlang bug tracker: Redirecting…

I’m pretty far removed from this issue at the moment, so may not have much further context to share, but I believe between this thread and the one in the bug tracker, everything I observed has been captured, in case it’s useful to anyone. Looking at the Erlang source code is likely provide a lot of clarity around where any blocking actually occurs, and echoing what has been said by a few folks in a few different places, it isn’t always what you might assume or expect in regards to durability, based on the documentation.

Where Next?

Popular in Questions Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lists...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; somethi...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement