nickewing

nickewing

Handling Phoenix Presence during testing

I have run into an issue while testing a simple Phoenix channel and using Phoenix.Presence.

It looks to me like the tests completes and the Presence process notices that a user has disconnected as a result. It then attempts to fetch the new user list but at that point the database connection has already been closed for that test. Is there any way to handle this sort of situation?

I tried calling Presence.unlink from within the test but was unable to get any other results.

I’m seeing the following error:

09:32:22.414 [error] Task #PID<0.580.0> started from MyApp.Presence terminating
** (stop) exited in: GenServer.call(#PID<0.577.0>, {:checkout, #Reference<0.0.5.2491>, true, 15000}, 5000)
    ** (EXIT) shutdown: "owner #PID<0.576.0> exited while client #PID<0.579.0> is still running with: shutdown"
    (db_connection) lib/db_connection/ownership/proxy.ex:32: DBConnection.Ownership.Proxy.checkout/2
    (db_connection) lib/db_connection.ex:919: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:741: DBConnection.run/3
    (db_connection) lib/db_connection.ex:584: DBConnection.prepare_execute/4
    (ecto) lib/ecto/adapters/postgres/connection.ex:80: Ecto.Adapters.Postgres.Connection.prepare_execute/5
    (ecto) lib/ecto/adapters/sql.ex:243: Ecto.Adapters.SQL.sql_call/6
    (ecto) lib/ecto/adapters/sql.ex:431: Ecto.Adapters.SQL.execute_and_cache/7
    (ecto) lib/ecto/repo/queryable.ex:130: Ecto.Repo.Queryable.execute/5
    (ecto) lib/ecto/repo/queryable.ex:35: Ecto.Repo.Queryable.all/4
    (my_app) lib/my_app/presence.ex:17: MyApp.Presence.fetch/2
    (phoenix) lib/phoenix/presence.ex:199: anonymous fn/5 in Phoenix.Presence.handle_diff/5
    (stdlib) lists.erl:1263: :lists.foldl/3
    (phoenix) lib/phoenix/presence.ex:197: anonymous fn/4 in Phoenix.Presence.handle_diff/5
    (elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: #Function<1.96140178/0 in Phoenix.Presence.handle_diff/5>
    Args: []

Using this channel module:

defmodule MyApp.ExampleChannel do
  use MyApp.Web, :channel

  def join(_channel_name, _params, socket) do
    send self(), :after_join

    {:ok, socket}
  end

  def handle_info(:after_join, socket) do
    current_user = socket.assigns.current_user

    {:ok, _} = Presence.track(socket, current_user.id, %{})

    {:noreply, socket}
  end
end

this presence module:

defmodule MyApp.Presence do
  import Ecto.Query
  alias MyApp.User
  alias MyApp.Repo

  use Phoenix.Presence, otp_app: :MyApp,
                        pubsub_server: MyApp.PubSub

  def fetch(_topic, entries) do
    query =
      from u in User,
        where: u.id in ^Map.keys(entries),
        select: {u.id, u}

    users = query |> Repo.all |> Enum.into(%{})

    for { key, %{ metas: metas } } <- entries, into: %{} do
      int_key = String.to_integer(key)
      { key, %{ metas: metas, user: users[int_key] } }
    end
  end
end

and this channel test:

defmodule MyApp.ExampleChannelTest do
  use MyApp.ChannelCase

  alias MyApp.ExampleChannel

  test 'example' do
    user = create_a_user

    params = %{ current_user: user }
    {:ok, _, socket} = subscribe_and_join(socket("", params), ExampleChannel, "control")
  end
end

Any help is greatly appreciated. Thanks!

Most Liked

rhcarvalho

rhcarvalho

Reviving the thread because a) there’s been more recent advancements that can help anyone landing here and b) I still have trouble making tests involving Phoenix.Presence deterministic.

In my particular case, my fetcher doesn’t use the DB but calls an external API, which I’m trying to mock with TestServer (context TestServer - No fuzz mocking of third-party services - #6 by rhcarvalho), but I believe the underlying trouble is the same.


First, summarizing the knowledge from the thread, in 2017 @luizpvasc and @sorentwo suggested something like:

ref = leave(socket)
assert_reply ref, :ok
:timer.sleep(10)

In late 2019, 2020, this GitHub issue brings more light into the problem:

https://github.com/phoenixframework/phoenix/issues/3619#issuecomment-601127561

Quoting José Valim:

The issue is that once the test terminates, the channel process will terminate, the presence process will notice the channel termination, and then invoke the callbacks without the database.

All of this happens async, so it is hard to make it sync. I am not sure at the moment how to fix those.

The conversation goes on and a new API has been added along with some docs:

In 2021, @ruslandoga notices something I’ve experienced as well in practice, that the new Presence.fetchers_pids() might pick up an empty list and so adds some sleep time before calling it:

https://github.com/phoenixframework/phoenix/issues/3619#issuecomment-782728849

on_exit(fn ->
    :timer.sleep(10)  #### WAIT FOR FETCHER PROCESSES TO BE STARTED ####
    for pid <- RumblWeb.Presence.fetchers_pids()  do
      ref = Process.monitor(pid)
      assert_receive {:DOWN, ^ref, _, _, _}, 1000
    end
  end)

I used GitHub Search to see what people are doing in the open: Code search results · GitHub

The first hit for me is LiveBeat which does use a 100ms sleep:

https://github.com/fly-apps/live_beats/blob/ac9780472e7019af274110a1cf71250a8d40c986/test/support/conn_case.ex#L39-L54

Second hit is NervesHub that just follows the documentation and has no sleep:

https://github.com/nerves-hub/nerves_hub_web/blob/2a8cd770e695cd518bfc76dc4b88f4ff08670bff/test/nerves_hub_web/live/new_ui/devices/show_test.exs#L27-L32

Going down the list I find both cases of with and without sleep, with different amounts of sleep. And found this commit message from @gpreston which reinforces people don’t know what to do :slight_smile: I don’t know what to do… the sleep time still feels non-deterministic, racy.

https://github.com/gcpreston/cuberacer_live/commit/5c77674f568dfb64151739d885352f28b30fcb7e


TODO:

  • I’m tempted to suggest updating the docs with the sleep before calling fetchers_pid
  • Discuss what else can we do. Could tests reasonable synchronize with Presence fetchers? Can we write tests knowing that a certain fetcher will be called exactly N number of times? Is that a bad idea to begin with?

Would love to learn more about this corner of Elixir. There are so many pieces involved in making Presence work that understanding how everything fits together (and points of synchronization) is no easy feat :slight_smile:

nickewing

nickewing

I never actually found a solution for this, but with our app we ended up removing the fetch function entirely. Now we just return user IDs and match them up with user data sent via other channels.

sorentwo

sorentwo

Oban Core Team

This is the correct solution in my experience. You must leave before the test is complete while the sandbox connection is still checked out—it also has to be synchronous to enable automatic sharing, as there isn’t any way to allow the presence process access to the test’s db connection.

The sleep doesn’t have to be nearly so long as 200ms though. The helper function I have uses a 10ms delay and doesn’t flicker on low powered systems like CI.

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New

Other popular topics Top

lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30877 112
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement