Trouble with Sandbox Mode for e2e tests

I’m having issues getting the sandbox plug working for my javascript based playwright tests.

I have a fairly standard and fresh phoenix app.

I wanted to create a test env similar to the standard test env, but with some tweaks for e2e tests, so I created a new config/test_ui.exs file that is a copy of the config/test.exs file but with server set to true:

import Config

# Only in tests, remove the complexity from the password hashing algorithm
config :bcrypt_elixir, :log_rounds, 1

# Configure your database
#
# The MIX_TEST_PARTITION environment variable can be used
# to provide built-in test partitioning in CI environment.
# Run `mix help test` for more information.
config :app, App.Repo,
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "app_test_ui#{System.get_env("MIX_TEST_PARTITION")}",
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: System.schedulers_online() * 2,
  loggers: [{Ecto.LogEntry, :log, [:debug]}]

# We don't run a server during test. If one is required,
# you can enable the server option below.
config :app, AppWeb.Endpoint,
  http: [ip: {127, 0, 0, 1}, port: 4002],
  secret_key_base: "2PQY5lqiEa9vguSZnFrprc9UrJ41GzYlSug+jskg42N+2RDSQCMPkZJ1bfddDbYv",
  server: true

# In test we don't send emails
config :app, App.Mailer, adapter: Swoosh.Adapters.Test

# Disable swoosh api client as it is only required for production adapters
config :swoosh, :api_client, false

# Print only warnings and errors during test
config :logger, level: :debug

# Initialize plugs at runtime for faster test compilation
config :phoenix, :plug_init_mode, :runtime

# Enable helpful, but potentially expensive runtime checks
config :phoenix_live_view,
  enable_expensive_runtime_checks: true

I added the ecto sandbox plug by following the docs here: Phoenix.Ecto.SQL.Sandbox — Phoenix/Ecto v4.6.3

plug Phoenix.Ecto.SQL.Sandbox,
  at: /sandbox,
  repo: App.Repo,
  timeout: 15_000,
  header: "x-app-sandbox"

I’m running it through a new mix alias:

test_ui: ["ecto.create --quiet", "ecto.migrate --quiet", "phx.server"]

Through logging, I can see that my playwright tests are properly calling /sandbox and Phoenix is getting the data back in the headers on future requests. All good there.

However, updates are being persisted to the DB as if Ecto were not in sandbox mode.

I suspect the issue is related to this line from the docs:

Finally, make sure your repository mode is set either to :manual or {:shared, self()} before the external client starts. This is typically done by default in your test/test_helper.exs, but you may need to do it explicitly depending on your setup:

I’m not sure where to set the sandbox to manual. Because I’m not using mix test the test_helper.exs isn’t running. I tried to create a test_ui_helper.exs file and call that in a few differet ways, but I keep getting the error:

02:03:27.781 [error] GenServer #PID<0.645.0> terminating
** (DBConnection.OwnershipError) cannot find ownership process for #PID<0.645.0>.

When using ownership, you must manage connections in one
of the four ways:

* By explicitly checking out a connection
* By explicitly allowing a spawned process
* By running the pool in shared mode
* By using :caller option with allowed process

The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.

The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.

The fourth option requires [caller: pid] to be used when
checking out a connection from the pool. The caller process
should already be allowed on a connection.

If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.

Manual mode means that you have to explicitly check out the connection on a per-process basis. If another process comes along and tries to access the database, it will fail because that process has not checked out a connection (see here in the docs - that’s your error).

Since you are running your Phoenix app for real (not as a test), each http request spawns a process, and that process has not checked out a sandbox connection manually.

I think the solution here depends on what you’re trying to do. If you just want to reset the DB after running all of the tests, you could just use shared mode and check out a single connection at the start of your tests. I think that means they would all block each other trying to check out the same connection, but they probably run serially anyway, right?

Of course, you could also just mix ecto.reset every time to get exactly the same result (an empty database), so perhaps that’s the simpler approach.

If you have multiple end to end tests and you want each one to be sandboxed, you’ll have to find some way to tell Phoenix to reset the shared connection after each test and then run them serially. You could also probably find a way to use manual mode and ensure each process has the proper allowance to run them in parallel, but I don’t see how you could do that in practice without a lot of unreasonable complexity.

1 Like

I believe the sandbox plug is supposed to take care of all of that out-of-the-box:

When I start a test, I make a POST to /sandbox which should start a transaction and return an encoded token to track on future requests. When the test is done, I make a DELETE to /sandbox to close the transaction.

Just creating config files won’t make them be used. Are you sure you run this in a way this is called?

The sandbox defaults to :auto mode, which is essentially “no sandboxing”. You’d want to call Ecto.Adapters.SQL.Sandbox — Ecto SQL v3.12.1 changing the mode before starting your playwright tests.

Just creating config files won’t make them be used. Are you sure you run this in a way this is called?

Correct, I’m running it with MIX_ENV=test_ui mix test_ui

The sandbox defaults to :auto mode, which is essentially “no sandboxing”. You’d want to call Ecto.Adapters.SQL.Sandbox — Ecto SQL v3.12.1 changing the mode before starting your playwright tests.

Right, I’ve tried a few different places to set the mode to manual, but then I get that error that I posted.

Apologies, I think my eyes glazed over reading the config and I missed that part of your post. That Plug does indeed do pretty much exactly what I was thinking!

You probably want to set the mode to manual when your application is started. The repo almost certainly has to be running, so you could do it at the end of start/2 after Supervisor.start_link(...) is called in your application.ex and see if that works. It would perhaps be more proper to start a one-off task in the supervision tree after the Repo.