Creating Mnesia tables with disc_copies option

Questions / Help
,
1

Hello, everyone

I’m trying to create a Mnesia table with using the mnesiam package, but can’t understand what the issue is happening during creation.

The table is very simple and doesn’t contain something special:

defmodule Matchmaking.Model.LobbyState do
  alias :mnesia, as: Mnesia

  @table Matchmaking.Model.LobbyState
  @attributes [
    :id,              # UUID
    :dump             # Shared state in JSON as string
  ]

  def init_store() do
    Mnesia.create_table(
      @table,
      [
        type: :set,
        disc_copies: [Node.self()],
        record_name: @table,
        attributes: @attributes
      ]
    )
  end

  def copy_store() do
    Mnesia.add_table_copy(@table, Node.self(), :disc_copies)
  end
end

If I’m trying to call the init_store/0 manually, then it returns the following record:

iex(1)> Matchmaking.Model.LobbyState.init_store()
{aborted: {:bad_type, Matchmaking.Model.LobbyState, :disc_copies, :nonode@nohost}

and :mnesia.system_info returns this:

iex(2)> :mnesia.system_info
===> System info in version "4.15.3", debug level = none <===
opt_disc. Directory "/app/mnesia_disc_dump" is NOT used.
use fallback at restart = false
running db nodes   = [nonode@nohost]
stopped db nodes   = [] 
master node tables = []
remote             = []
ram_copies         = ['Elixir.Matchmaking.Model.ActiveUser',schema]
disc_copies        = []
disc_only_copies   = []
[{nonode@nohost,ram_copies}] = [schema,'Elixir.Matchmaking.Model.ActiveUser']
3 transactions committed, 7 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
:yes

Finding the answer to this issue led to the adding a variable in configs (for using disc_copies tables you will need to define a directory for storing dumps). However, an attempt to specify a directory in config.exc :

config :mnesia, :dir, System.get_env("MNESIA_DUMP_DIRECTORY") || '/app/mnesia_disc_dump'

with rights for writing/reading/executing didn’t help me. The expected table still isn’t created and returns the same error as was recently mentioned.

Anybody had faced with the same issue before? How it can be solved?

2 Likes
2

A first question: did you create a schema before starting mnesia and creating your table? When you create a schema you create a directory structure where all mnesia data is kept. Disc copies of tables are kept in this directory. If you don’t explicitly create a schema then everything is kept in RAM so creating a disc doesn’t seem reasonable.

You must create the schema before starting mnesia and you only have to create the schema the first time, after that it will be reused.

1 Like
3

This stuff is executing by Mnesiam package. Currently I’m trying to run the Mnesia node alone with the code from that library:

defmodule Mnesiam do
  @moduledoc """
  Mnesia Manager
  """
  require Logger
  alias Mnesiam.Store
  alias :mnesia, as: Mnesia

  @doc """
  Start Mnesia alone
  """
  def start do
    with :ok <- ensure_dir_exists(),  # creates the directory if it doesn't exist
         :ok <- Store.init_schema(),  # creates schema for the node, **before** starting an app
         :ok <- start_server(),  # running Mnesia app
         :ok <- Store.init_tables(),  # calling `init_store/0` function per each specified module in config
         :ok <- Store.ensure_tables_loaded() do
      :ok
    else
      {:error, error} -> {:error, error}
    end
  end

...
end

Where is the init_schema/0 is doing the following stuff:

def init_schema do
    current_node = Node.self()
    case Mnesia.system_info(:extra_db_nodes) do
      [] ->
        case Mnesia.create_schema([current_node]) do
          :ok -> :ok
          {:error, {_, {:already_exists, _}}} -> :ok
          {:error, reason} -> {:error, reason}
        end
      [_|_] -> :ok
    end
end

And after it, after the init step, the Mnesiam application is trying to run Mnesia with start_server/0 call. Therefore, we make sure in that the project is doing that:

$ iex -S mix
Interactive Elixir (1.6.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> alias :mnesia, as: Mnesia
:mnesia
iex(2)> Mnesia.system_info(:directory)
'/app/mnesia_disc_dump'
iex(3)> File.exists?(Mnesia.system_info(:directory))
true
iex(4)> Mnesia.system_info(:is_running)
:yes
iex(5)> Mnesia.create_schema([Node.self()])
{:error, {:nonode@nohost, {:already_exists, :nonode@nohost}}}
iex(6)> Mnesia.system_info(:extra_db_nodes)
[]
iex(7)> Mnesiam.Store.init_schema()
:ok
iex(8)> Mnesiam.Store.init_tables()
:ok

Probably it is a bug in Mnesiam package, but I’m not sure. If anyone else was faced with it before, please, let me know.

4

I’m facing this exact issue right now, not using any abstraction and instead directly working with :mnesia. It works just fine on my desktop, but as soon as I deploy it to nerves I’m getting the same issue. Currently investigating it. Please tell if you found a solution. :slight_smile: Thanks!

5

I was encountering a similar issue trying to use Mnesia for persistent sessions with Pow. I have a full breakdown of the situation, the problems I encountered, and my solution here.

To summarize, I am using Edeliver/Distillery, and so I had added mnesia to my extra_applications, and this was starting :mnesia with the default schema in memory, then Pow was trying to create another schema on the same node, but with :disk_copies enabled. This lead to the same :bad_type error as @Relrin was getting.

To fix this, I removed mnesia from :extra_applications, and instead put it in :included_applications, so that Mnesia is available to my app in production, but no default schema is created.

5 Likes
6

Great that you got it working, and thanks for writing a guide that will help others in the same situation!

There is also a resolved issue on Github. From personal experience Mnesia errors are mostly just initialization that goes wrong, and then there may be subsequent errors if the generated disk copy files weren’t removed before deploying a fixed version.

I do recommend setting up mnesia both in dev and prod to keep them aligned as much as possible :smile:

Elixir 1.9.0 will be out soon so I plan to update the Pow docs with instructions how to get Pow running with mix release.

BTW, in your application module you could look up the :cache_store_backend so you won’t have to set the :pow_session_cache_worker config. Could look something like this:

defmodule MyApp.Application do
  use Application
  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      supervisor(MyApp.Repo, []),
      supervisor(MyAppWeb.Endpoint, []),
    ] ++ workers()

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end

  def config_change(changed, _new, removed) do
    MyAppWeb.Endpoint.config_change(changed, removed)
    :ok
  end

  defp workers() do
    import Supervisor.Spec

    case Application.get_env(:my_app, :pow)[:cache_store_backend] do
      Pow.Store.Backend.MnesiaCache -> [worker(Pow.Store.Backend.MnesiaCache, [[nodes: [node()]]])]
      _ -> []
    end
  end
end
7

Wow, it’s incredible that that Github issue didn’t come up in any of my searches. Thanks for that (and for the refactor too)!

Initially was also thinking that mnesia should be setup in both dev and prod. I think I misunderstood the first line of the Pow readme cache store section - the part about the EtsCache “can be used in development and test environment” I misinterpreted as “should be used for development and test.”

1 Like
8

Weird, when I try to do this I get an error.

** (Mix) Undefined applications: [mnesia]
9

I’m not 100% sure, but I think this is one of the many errors I came across when doing an exploratory upgrade to Elixir 1.9. Are you using 1.9? If so, you may want to try downgrading to 1.8 and see if that works for you.

1 Like
10

Yes, 1.9. I wasted a whole day trying to get it to work yesterday, and rewrote to just use ETS last night.

I’m done with mnesia for the foreseeable future. :rage:

11

It’s probably not an issue with mnesia, but just with :included_applications not being used in releases in Elixir 1.9.0 (fixed in 1.9.1): https://github.com/elixir-lang/elixir/issues/9163

Mnesia has been working pretty well for me personally. Just wished the documentation and error messages was better.

2 Likes