Phoenix testing with Ecto 2 sandbox - access from processes

Questions / Help
, ,
1

I am testing a channel in a Phoenix app using the generated ChannelCase, which uses the shared sandbox mode. My channel wants to fetch some data from a GenServer running in the app’s supervision tree. That GenServer gets loaded with data from the database at startup, and I have a reset function to reload it during testing.

In my test, I insert some fixture data into the database, then reset the GenServer with the intention that it gets loaded with the fixture data. Then I do a channel test to make sure that I get the expected behavior.

What is happening, however, is that the GenServer is not getting any of the fixtures that I’ve inserted from the test. I can observe the queries happening in the logs, and log out the results of the inserts and queries to confirm that the fixtures are being inserted but the query is returning nothing.

I assume this has something to do with the connection ownership of the Sandbox mode? Is there a relatively simple way to get around this? I don’t need the tests to run concurrently, but I do want the state to be reset after each test. I’m sure I can find some kind of workaround, but I’d like to understand why this isn’t working. I have written tests similar to this before that worked, so I’m also confused about what could be different…

I’m using Ecto 2.2.6, PhoenixEcto 3.3.0, Phoenix 1.3.0

2

I’m still having no luck here, and I’m even more confused because I have looked back at other projects using the same versions of Ecto / Phoenix / etc. where I have used this pattern.

This is an open source project if looking at the code would help: https://github.com/dantswain/cooky. It’s a demo application for a presentation on using channels and OTP.

I wrote two “reset” functions for the GenServer that I’m interacting with:

  # makes the database query from the caller
  def reset do
    ingredients = Cooking.all_ingredients()
    GenServer.call(__MODULE__, {:reset, ingredients})
  end

  # makes the database query from inside the genserver
  def reset_in_proc do
    GenServer.call(__MODULE__, :reset)
  end

The reset/0 function works with the tests, the reset_in_proc/0 function does not. This is what my supervision tree looks like:

    # Define workers and child supervisors to be supervised
    children = [
      # Start the Ecto repository
      supervisor(Cooky.Repo, []),
      # Start the endpoint when the application starts
      supervisor(CookyWeb.Endpoint, []),
      worker(Cooky.Chef, [])
    ]

I’m using ChannelCase unmodified.

3

Do you see any errors in test output? A minimal project that reproduces behavior and can be cloned will help other to help you.

4

@astery Yeah, the link is in my reply. To reproduce the error, edit cooking_channel_test.exs and change

    # fails
    # Chef.reset_in_proc()
    Chef.reset

to

    # fails
    Chef.reset_in_proc()
    # Chef.reset

The actual test error is a MatchError that’s deeper down in the code and due to the query not returning any rows. The only difference between failing and not failing is where the database query is performed - in the test process (works) or in the GenServer (fails).

5

I’m not sure why genserver process needs to be started after shared mode turned on, and why in this case even if we turn off it no error printed. It’s interesting for me to know.

Sure a better workaround should exist, but we can just restart genserver after shared mode is on (any way at some point you will want to do it to reset it’s state).

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(Cooky.Repo)

    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(Cooky.Repo, {:shared, self()})
    end

    :ok = Supervisor.terminate_child(Cooky.Supervisor, Cooky.Chef)
    {:ok, _} = Supervisor.restart_child(Cooky.Supervisor, Cooky.Chef)
    :ok
  end

And it will cause to Repo to send queries in same transaction/connection as test.

1 Like
6

@astery Hm. That’s interesting. I also tried putting the worker in a separate supervisor, but that doesn’t work. I have other applications that use this same pattern and tests work as expected… I’d really like to figure out what the difference is :confused:

7

Without the applications, it will be hard to know. But whenever you need to collaborate with process that already exist, it is most likely that you will need to set async: false to avoid race conditions.

8

Thanks, @josevalim. There is a link to the source above. It’s a demo for a presentation and not very complex. Everything is out-of-the-box Phoenix unless I have inadvertently changed something. async is set to false (I’m fine with these tests being asynchronous).

9

Oh, nice. What do I need to reproduce this then? Just git clone and mix test? Also, if you are getting an error, can you please paste the full error and stacktrace? Thanks.

10

I’ve simplified interesting part

git clone https://github.com/astery/cooky
git co test
mt test/interesting_test.exs
11

The issue is the initial query that you do on GenServer.init. This makes a connection to be allocated to the GenServer and then it doesn’t pick the shared one.

I have opened an issue here: https://github.com/elixir-ecto/db_connection/issues/96

For now, one alternative is to lazily start the gen server. Something like:

def init(arg) do
  {:ok, {:lazy, arg}}
end

def handle_call(msg, from, {:lazy, arg}) do
  handle_call(msg, from, State.init(...))
end
2 Likes
12

Another idea is to do:

Supervisor.terminate_child(YourMainSupervisor, :name_of_that_process)
Supervisor.restart_child(YourMainSupervisor, :name_of_that_process)

after you start the sandbox.

13

Thanks, @astery and @josevalim ! This makes sense. I’m sort of glad to find out it was a bug and not just me going crazy ;).