Testing Chunked responses from a phoenix endpoint using `Plug.Test` or `Phoenix.ConnTest`

Phoenix Forum Questions / Help
,
1

I am implementing Server Sent events in a phoenix project. This is easy enough to do by using plug directly. I am following this gist.

However I am not sure of the best way to test such an endpoint. I could use HTTPoison Async Requests and connect to the test server but I would rather just test the conn object using Plug.Test or Phoenix.ConnTest. Is this possible are there any examples out there on how to do this.

2 Likes
2

I researched this a bit, and it’s indeed far from being trivial. From what I found, when we call Plug.Conn.chunk(conn, chunk), plug calls conn.adapter.chunk. From there, the chunk should be sent to the server (e.g. cowboy) for further handling. The conn is not aware of the chunk anymore.

In tests, the adapter is Plug.Adapters.Test.Conn, and its chunk function reply with {:ok, payload}. It doesn’t really matter what the payload is, because Plug.Conn.chunk is probably called from a process that is not going to finish / return this state in any way. This process communicate messages back to the client forever, in a loop.

So, in the case of the test, I see no way to get the chunk out of the process, at least not in a “conventional” way. So, I tried to solve this by mocking Plug.Conn.chunk to send a the chunk to the test process, and there I assert_receive for it. You can see an example here. It’s not the cleanest solution, I know… What do you think?

1 Like
3

I tried to get the method by @Nagasaki45 but I could not get it to work. I also tried mocking Plug.Conn.chunk which also didn’t work for me I ended up doing this.

I created a function with minimal side effects and used that to run the chunking.

defmodule MyApp.ControllerUtils do
    use MyaAppWeb, :controller

    @callback chunk_to_conn(map(), String.t()) :: map()
    def chunk_to_conn(conn, current_chunk) do
        conn |> chunk(current_chunk)
    end
end

Now in my controller, i call chunk_to_conn(conn, current_chunk) which I can mock in test to give me the chunks like this
In my test support

Mox.defmock(MyAppWeb.ControllerUtilsMock, for: MyAppWeb.ControllerUtils)

And in my test

defmodule MyApp.MyTest do
   import Mox
   defp chunked_response_to_state(chunk, pid) do
    current_chunk = Agent.get(pid, &Map.get(&1, :chunk_key))
    Agent.update(pid, &Map.put(&1, :csv, current_chunk <> chunk))
   end
   setup do
       MyApp.ControllerUtilsMock
         |> stub(:chunk_to_conn, fn _, chunk -> chunk |> chunked_response_to_state(agent_pid) end)
        {:ok, %{agent_pid: agent_pid}}
   end
   test "my test", state do
       build_conn.get(somepath)
       whole_chunks = Agent.get(state.agent_pid, &Map.get(&1, :chunk_key))
   end

end
1 Like
4

I am resurrecting this topic to share that you can test chunk responses if you follow the documentation for chunk. The test adapter collects chunks and updates the response body.

defp send_csv_response(%Plug.Conn{} = conn, _) do
  {:ok, %Plug.Conn{state: :chunked} = conn} =
    conn
    |> put_resp_content_type("text/csv")
    |> put_resp_header("content-disposition", ~s|attachment; filename="foo.csv"|)
    |> send_chunked(:ok)

  Enum.reduce_while(["a", "b", "c"], conn, fn data, conn ->
    case chunk(conn, data) do
      {:ok, conn} -> {:cont, conn}
      {:error, :closed} -> {:halt, conn}
    end
  end)
end

And in tests

test "chunked response", %{conn: conn} do
  %Plug.Conn{state: :chunked} = response = conn |> get(page_path(conn, :send_csv_response)
  content = response(response, 200)
  assert content == "abc"
end
5

The setup you’re describing works because the chunks are being sent before your controller returns the conn. So your test only gets a hold of the response after all chunks have been sent. If any of the chunks are being sent asynchronous, this won’t work. Your controller doesn’t really trigger that situation.

I’ve come across the situation where the chunks are being streamed from a transaction, which takes some time. Then the chunks are only sent after the controller has returned the conn. In that case, the chunks are not available yet in the test.

I resorted to using a real http client (req) from tests to obtain the complete response (req then does the waiting and accumulating of the whole body). It required some fiddling with cookies to ensure the request was authorized though.