Exactly that, yes. It has been a long road to get to this point TBH. I had problems with test data (I tried ex_machina, and then factori, before finding seed_factory). Then I had problems with various caching stuff. My final problem was the issue of “connected mount”. I just wanted to share my (hacky) solution to how I solved it.
If I get time, I will try the link provided above by @LostKobrakai , although it makes no mention of affording async tests.
I’d just not start those workers (globally). They’re not really useful anyways because the test db is never meant to hold data anyways. For testing the workers you can start them within the tests as described here: Ecto.Adapters.SQL.Sandbox — Ecto SQL v3.12.1
Thanks for the information. I would rather test this app as it is meant to run in reality. Every app is different and there is no perfect approach for every app.
Please let me know if the link you posted several comments ago affords running ConnCase tests with async: true. Thanks!
I’d say generally no. No solution will be able to do that. A worker started by the application acts on its own unrelated to an individual test, so which tests sandbox does it use? If you find some answer to that: What if that worker stores some internal state based on what it found in the db, now a test starts interacting with it. There’s a non-zero chance the internal state of the worker doesn’t match the state it’s supposed to hold base on the sandbox of the test. If it magically matches, what if multiple concurrent tests interact with the worker expecting it to hold distinct internal state related to each their active sandboxes.
In short: You cannot make a singleton – your one instance of globally started application with all its processes in the supervision tree – support arbitrary concurrent independent state handling.
The optimal state would be having a version of your application started per test. Given that’s often a lot of work and hard to coordinate the answer to that problem is commonly to test parts of that system in an concurrent fashion, while leaving testing of the assembled whole to integration/acceptance tests, which are fewer in number and do not run concurrently.
Even the link I posted above works only as far as scoping a single “user” on http requests/channels/LV to a sandbox, which may extend to further spawned processes, but once you get to globally spawned processes you again run into the problems I wrote in this post.
Sorry for the almost necrobump but I’ve explored the initial approach and made it work more seamlessly with Ecto’s Sandbox.
In a custom ExUnit case I’d add the sandbox metadata to the session.
setup tags do
pid = Ecto.Adapters.SQL.Sandbox.start_owner!(MyApp.Repo, shared: not tags[:async])
on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(MyApp.Repo, self())
conn =
Phoenix.ConnTest.build_conn()
|> Plug.Test.init_test_session(%{})
|> Plug.Conn.put_session(:sandbox, metadata)
{:ok, conn: conn}
end
Then, in a on_mount hook I’d use the session data to allow the LiveView process to use the sandbox.
defmodule MyAppWeb.Sandbox do
@moduledoc false
import Phoenix.LiveView, only: [connected?: 1]
def on_mount(:default, _params, session, socket) do
if connected?(socket) do
%{repo: repo, owner: owner} = get_in(session["sandbox"])
Ecto.Adapters.SQL.Sandbox.allow(repo, owner, self())
end
{:cont, socket}
end
end
The rest of the setup is the same as in the original post. Of course, one needs to have the Ecto’s sandbox properly set up, following the documentation.
I am really happy that at least 1 person found it useful!
What you have done is definitely an improvement. Thank you.
Thank you for the kind words. Unfortunately, I hit a problem. If we use with the ownership model, we need to allow every spawned process, which needs to use the database, to use the sandbox. For example, if I use assign_async, in the aforementioned hook, I need to put a private assign holding the sandbox metadata and write a wrapper like the following.
def assign_async_sandboxed(
%{private: %{sandbox: %{owner: owner, repo: repo}}} = socket,
name,
fun
) do
assign_async(socket, name, fn ->
# Works because the thunk is called in the newly created process.
Ecto.Adapters.SQL.Sandbox.allow(repo, owner, self())
fun.()
end)
end
I imagine in a greenfield project this could work good but probably not otherwise. I was thinking that maybe we’re not focusing on the right thing. It might be more important to provide a stubbed implementation of the business logic instead of going great lengths to allow the Ecto sandbox. Unfortunately, we don’t have control over how the LiveView process is spawned and can’t adequately inject dependencies.
If assign_async doesn’t propagate ownership I’d consider this a bug. Afaik async sits on top if Tasks so I’m not even sure how that could be messed up with Task supporting ownership ootb.
This definitely is a problem with my approach.
The only thing I can think to do to fix it is to make wrapper functions around the async live view functions and then write a credo rule to make sure nobody calls assign_async and start_async in the code. 