TestServer - No fuzz mocking of third-party services

rhcarvalho · May 27, 2025, 6:59am

Tests are run synchronously, and the issue is reproducible running a single test case in isolation as well.

I could be doing something wrong in the “testing Phoenix.Presence” side of things. I’ve read the few threads on the topic I could find in this forum, including Handling Phoenix Presence during testing - #3 by luizpvasc. I originally worked on this already some time ago, memory is a bit vague, but I think it is related to how Presence “fetchers” work asynchronously, process linking and termination.

So I considered posting on that topic, but then thought maybe TestServer could just be less strict about the number of calls. Reading the implementation a bit more last night, I believe that’s not possible, though.

github.com/danschultzer/test_server

lib/test_server.ex

21ec25c4c


      
          def start(options \\ []) do
            case ExUnit.fetch_test_supervisor() do
              {:ok, sup} ->
                start_with_ex_unit(options, sup)
          
              :error ->
                raise ArgumentError, "can only be called in a test process"
            end
          end

github.com/danschultzer/test_server

lib/test_server.ex

21ec25c4c


      
          defp start_with_ex_unit(options, _sup) do
            [_first_module_entry | stacktrace] = get_stacktrace()
          
            case InstanceManager.start_instance(self(), stacktrace, options) do
              {:ok, instance} ->
                put_ex_unit_on_exit_callback(instance)
          
                {:ok, instance}
          
              {:error, error} ->
                raise_start_failure({:error, error})
            end
          end

github.com/danschultzer/test_server

lib/test_server.ex

21ec25c4c


      
          defp put_ex_unit_on_exit_callback(instance) do
            ExUnit.Callbacks.on_exit(fn ->
              case Process.alive?(instance) do
                true ->
                  verify_routes!(instance)
                  verify_websocket_handlers!(instance)
                  stop(instance)
          
                false ->
                  :ok
              end
            end)
          end

github.com/danschultzer/test_server

lib/test_server.ex

21ec25c4c


      
          defp verify_routes!(instance) do
            instance
            |> Instance.routes()
            |> Enum.reject(& &1.suspended)
            |> case do
              [] ->
                :ok
          
              active_routes ->
                raise """
                #{Instance.format_instance(instance)} did not receive a request for these routes before the test ended:
          
                #{Instance.format_routes(active_routes)}
                """
            end
          end

TestServer always sets up an ExUnit on_exit callback that checks routes: if anything is left unvisited it errors out.

And when an unexpected request comes, it produces the warning:

github.com/danschultzer/test_server

lib/test_server/plug.ex

21ec25c4c


      
          {:error, {:not_found, conn}} ->
            message =
              "#{Instance.format_instance(instance)} received an unexpected #{conn.method} request at #{conn.request_path}"
              |> append_formatted_params(conn)
              |> append_formatted_routes(instance)

I think either I need to understand how to test Phoenix.Presence in a more deterministic way, spin up a dumb server that doesn’t assert on the number of mocked calls, or mock at the HTTP client level.