Genserver doesn't consider Ecto Sandbox

Hi, I have a test function like this:

test "test one" do
  depends = ["joomla_login_plugin", "wordpress_login_plugin", "magento_login_plugin"]
  
  # Create data in db
  Enum.map(depends, fn item ->
    Map.merge(@new_soft_plugin, %{name: item, depend_type: :hard, depends: List.delete(depends, item)})
    |> Map.from_struct()
    |> Plugin.create()
  end)
  
  # Create state for each data which was imported to db
  Enum.map(depends, fn item ->
    Map.merge(@new_soft_plugin, %{name: item, depend_type: :hard, depends: List.delete(depends, item)})
    |> MishkaInstaller.PluginState.push()
  end)

end

If my state is like this, I have no problem:

  @impl true
  def handle_cast({:push, status, %PluginState{} = element}, state) do
    {:noreply, element, state}
  end

this state is created without any problem, but if I pass this state to the handle_continue for adding or editing it to my db like this:

@impl true
def handle_cast({:push, status, %PluginState{} = element}, _state)
  {:noreply, state, {:continue, {:sync_with_database, :add}}}
end

@impl true
def handle_continue({:sync_with_database, _status}, %PluginState{} = state) do
  state
  |> Map.from_struct()
  |> Plugin.add_or_edit_by_name()
  |> check_output(state)
  {:noreply, state}
end

it is stored and if I run mix test again, I can see the records still exist in test db, but in the top of my test module I added Sandbox like this:

setup do
  :ok = Ecto.Adapters.SQL.Sandbox.checkout(MishkaDatabase.Repo)
end

This is my config:

  config :mishka_database, MishkaDatabase.Repo,
    username: System.get_env("DATABASE_USER"),
    password: System.get_env("DATABASE_PASSWORD"),
    database: "#{System.get_env("DATABASE_NAME")}_test#{System.get_env("MIX_TEST_PARTITION")}",
    hostname: System.get_env("DATABASE_HOST"),
    show_sensitive_data_on_connection_error: true,
    pool: Ecto.Adapters.SQL.Sandbox

By the way, my project is umbrella

Thank you, let me know how I can fix this

If you generate a new phoenix project, you can see the file /test/support/data_case.ex, this is used currently to run tests with database interaction, the file contains a custom setup:

setup tags do
    pid = Ecto.Adapters.SQL.Sandbox.start_owner!(DetectionsReporter.Repo, shared: not tags[:async])
    on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
    :ok
  end

That’s probably not the cause of the issue here though.

@shahryarjb consider the docs here: Ecto.Adapters.SQL.Sandbox — Ecto SQL v3.7.2

1 Like

When I use this, I got error in all my projects which were tested before it. I think I need to stop Ecto connection after the test, but I do not know how

13:37:38.582 [error] Task #PID<0.2073.0> started from #PID<0.2052.0> terminating
** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.2073.0>.

When using ownership, you must manage connections in one
of the four ways:

* By explicitly checking out a connection
* By explicitly allowing a spawned process
* By running the pool in shared mode
* By using :caller option with allowed process

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

The fourth option requires [caller: pid] to be used when
checking out a connection from the pool. The caller process
should already be allowed on a connection.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

By the way, my project is an elixir umbrella which has phoenix project and some normal elixir project under itself and my database and Ecto is in a specific elixir project not a phoenix one

I am not sure, it works!! But the another projects have many warning, I think they lose their connection.

I tried to add it in another test fields but it didn’t work and I have error like this:

 41) test Happy | MishkaApi Content Controller (▰˘◡˘▰) create tag (MishkaApiWeb.ContentControllerTest)
     apps/mishka_api/test/mishka_api_web/controllers/content_contoller_test.exs:631
     ** (MatchError) no match of right hand side value: {:error, {{:badmatch, {:already, :owner}}, [{Ecto.Adapters.SQL.Sandbox, :"-start_owner!/2-fun-0-", 3, [file: 'lib/ecto/adapters/sql/sandbox.ex', line: 411]}, {Agent.Server, :init, 1, [file: 'lib/agent/server.ex', line: 8]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 423]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 390]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}}
     stacktrace:
       (ecto_sql 3.7.2) lib/ecto/adapters/sql/sandbox.ex:403: Ecto.Adapters.SQL.Sandbox.start_owner!/2
       test/mishka_api_web/controllers/content_contoller_test.exs:28: MishkaApiWeb.ContentControllerTest.__ex_unit_setup_2/1
       test/mishka_api_web/controllers/content_contoller_test.exs:1: MishkaApiWeb.ContentControllerTest.__ex_unit__/2

I am using dynamic supervisor and registry, so I need to call my main function like this:

  def push(%PluginState{} = element) do
    case PSupervisor.start_job(%{id: element.name, type: element.event}) do
      {:ok, status, pid} -> GenServer.cast(pid, {:push, status, element})
      {:error, result} ->  {:error, :push, result}
    end
  end

But I used this, what the document said. PluginStateOtpRunner is the stuff I added it on my Application

plugin_runner_config = [
      strategy: :one_for_one,
      name: PluginStateOtpRunner
    ]
    children = [
      {Registry, keys: :unique, name: PluginStateRegistry},
      {DynamicSupervisor, plugin_runner_config}
    ]

and my test code

test "delete plugins dependencies with dependencies which exist" do
      allow = Process.whereis(PluginStateOtpRunner)
      Ecto.Adapters.SQL.Sandbox.allow(MishkaDatabase.Repo, self(), allow)

      depends = ["joomla_login_plugin", "wordpress_login_plugin", "magento_login_plugin"]
      Enum.map(depends, fn item ->
        Map.merge(@new_soft_plugin, %{name: item, depend_type: :hard, depends: List.delete(depends, item)})
        |> Map.from_struct()
        |> Plugin.create()
      end)
      |> IO.inspect()

      Enum.map(depends, fn item ->
        Map.merge(@new_soft_plugin, %{name: item, depend_type: :hard, depends: List.delete(depends, item)})
        |> MishkaInstaller.PluginState.push()
      end)
      |> IO.inspect()
    end

My Genserver always can find my db and has no problems, after running

MIX_ENV=test mix ecto.drop; MIX_ENV=test mix ecto.create; MIX_ENV=test mix ecto.migrate

I have no problem because my db is clean, but for second time, the data was stored in first test was not cleaned and still exist in db.

I can clean my database with running repo.delete manually, but it is a wasting time and I need to find what is my problem

for example

  {:error, :add, :plugin,
   #Ecto.Changeset<
     action: :insert,
     changes: %{
       depend_type: :hard,
       depends: ["joomla_login_plugin", "wordpress_login_plugin"],
       event: "event_one",
       extra: [],
       name: "magento_login_plugin",
       priority: 1,
       status: :started
     },
     errors: [
       name: {"duplicated record",
        [constraint: :unique, constraint_name: "index_plugins_on_name"]}
     ],
     data: #MishkaDatabase.Schema.MishkaInstaller.Plugin<>,
     valid?: false
   >}

I expect after running each test macro it flashes and cleans the data

Hi again, I add this line top of my test module:

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(MishkaDatabase.Repo)
    pid = Ecto.Adapters.SQL.Sandbox.start_owner!(MishkaDatabase.Repo, shared: not tags[:async])
    # on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
    :ok
  end

I disabled stop_owner and all the lines of my tests are run without errors and every test is started with a clean db, not like the previous examples

But I wonder, where should I run Ecto.Adapters.SQL.Sandbox.stop_owner(pid) to close this, it is necessary to stop_owner??!! All the example and new project in phoenix and documents which I found say put it after start_owner.
With stop_owner, I have many errors in sub-projects under umbrella. By the way, I removed Ecto.Adapters.SQL.Sandbox.mode(Repo, :manual) from test_helper file, it did not let the another sub-projects start without errors.

it should be noted I can not run test as use ExUnit.Case, async: true, I have error with async

  1) test Happy | Plugin Database (▰˘◡˘▰) delete plugins dependencies with dependencies which exist (MishkaInstallerTest.State.PluginDatabaseTest)
     apps/mishka_installer/test/plugin_manager/state/plugin_database_test.exs:52
     ** (MatchError) no match of right hand side value: {:error, {{:badmatch, {:already, :owner}}, [{Ecto.Adapters.SQL.Sandbox, :"-start_owner!/2-fun-0-", 3, [file: 'lib/ecto/adapters/sql/sandbox.ex', line: 411]}, {Agent.Server, :init, 1, [file: 'lib/agent/server.ex', line: 8]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 423]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 390]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}}
     stacktrace:
       (ecto_sql 3.7.2) lib/ecto/adapters/sql/sandbox.ex:403: Ecto.Adapters.SQL.Sandbox.start_owner!/2
       test/plugin_manager/state/plugin_database_test.exs:20: MishkaInstallerTest.State.PluginDatabaseTest.__ex_unit_setup_0/1
       test/plugin_manager/state/plugin_database_test.exs:1: MishkaInstallerTest.State.PluginDatabaseTest.__ex_unit__/2

Any suggestion?
Thank you.


Update

it works when there is just one test, if you have more than one it shows you top error Genserver doesn't consider Ecto Sandbox - #6 by shahryarjb, and you should activate stop_owner, but after activating it, another projects tests have error

I am done :)))

Temporary solution

  setup_all _tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(MishkaDatabase.Repo)
    Ecto.Adapters.SQL.Sandbox.mode(MishkaDatabase.Repo, :auto)
    on_exit fn ->
      :ok = Ecto.Adapters.SQL.Sandbox.checkout(MishkaDatabase.Repo)
      Ecto.Adapters.SQL.Sandbox.mode(MishkaDatabase.Repo, :auto)
      clean_db()
      :ok
    end

    [this_is: "is"]
  end

If I find disadvantage of this block code, I’ll update this page, for now I tested in a file with multi tested and I had no problem