How to fix "Plug.Conn.AlreadySentError" when running tests in Phoenix?

Phoenix Forum Questions / Help
1

I’m getting a “Plug.Conn.AlreadySentError” when running tests in my Phoenix application. Specifically, the error message says:

test lists all user (DbserviceWeb.UserControllerTest)
test/dbservice_web/controllers/user_controller_test.exs:69
** (Plug.Conn.AlreadySentError) the response was already sent
code: conn = get(conn, Routes.user_path(conn, :index))

How can I fix this error? Here’s my test code:

 setup %{conn: conn} do
    {:ok, conn: put_req_header(conn, "accept", "application/json")}
  end

  describe "index" do
    test "lists all user", %{conn: conn} do
      conn = get(conn, Routes.user_path(conn, :index))
      [head | _tail] = json_response(conn, 200)
      assert Map.keys(head) == @valid_fields
    end
  end

I guess this happening because there is already a get method being used inside the Conn module, and calling get again on the same connection causes the "AlreadySentError.

I understand that this error occurs when the response has already been sent before the test can complete. However, I’m not sure how to fix it. Can someone please help me understand what’s causing this error and how to fix it?

2

Can you post the full stack trace and/or the source of UserController? Usually this error happens if you try to render after redirecting or similar.

3

sure , Hey @al2o3cr Just to confirm, when I do IO.inspect (conn) before this line of code: conn = get(conn, Routes.user_path(conn, :index)) there is already a get method being used inside the Conn module, and I guess calling get again on the same connection causes the "AlreadySentError.

Here is my usercontroller code


  swagger_path :index do
    get("/api/user")
    response(200, "OK", Schema.ref(:Users))
  end

  def index(conn, params) do
    param = Enum.map(params, fn {key, value} -> {String.to_existing_atom(key), value} end)
    user =
      Enum.reduce(param, User, fn
        {key, value}, query ->
          from u in query, where: field(u, ^key) == ^value

        _, query ->
          query
      end)
      |> Repo.all()

    render(conn, "index.json", user: user)
  end
4

What is the intent behind this block in your user controller? It is most likely the culprit.

5

The intent behind this code is to specify the details of the API endpoint, including the HTTP method (GET ), the URL path (/api/user ), and the expected response schema (Users ).

can you help me to remove this error (Plug.Conn.AlreadySentError) the response was already sent

6

also Here is full stack trace of error

 test index lists all user (DbserviceWeb.UserControllerTest)
     test/dbservice_web/controllers/user_controller_test.exs:72
     ** (Plug.Conn.AlreadySentError) the response was already sent
     code: conn = get(conn, Routes.user_path(conn, :index))
     stacktrace:
       (phoenix 1.6.10) lib/phoenix/controller.ex:529: Phoenix.Controller.put_new_layout/2
       (dbservice 0.1.0) lib/dbservice_web/controllers/user_controller.ex:1: DbserviceWeb.UserController.phoenix_controller_pipeline/2
       (phoenix 1.6.10) lib/phoenix/router.ex:354: Phoenix.Router.__call__/2   
       (dbservice 0.1.0) lib/dbservice_web/endpoint.ex:1: DbserviceWeb.Endpoint.plug_builder_call/2
       (dbservice 0.1.0) lib/dbservice_web/endpoint.ex:1: DbserviceWeb.Endpoint.call/2
       (phoenix 1.6.10) lib/phoenix/test/conn_test.ex:225: Phoenix.ConnTest.dispatch/5
       test/dbservice_web/controllers/user_controller_test.exs:73: (test)      


Finished in 0.3 seconds (0.00s async, 0.3s sync)
1 test, 1 failure
7

The stack trace shows that this function is raising the error:

The usual cause of AlreadySentErrors from the Plug machinery like this is a previous plug that renders or redirects, but does not call halt to stop the pipeline. A common places for this to happen is in authentication or authorization plugs, do you have any of those involved?

1 Like
8

@al2o3cr Yes , there is domain_whitelist_plug.ex


  import Plug.Conn

  def init(options) do
    options
  end

  def call(conn, _options) do
    if allowed_domains?(conn) do
      conn
    else
      send_resp(conn, 403, "Not Authorized")
    end
  end

  defp allowed_domains?(conn) do
    whitelisted_domains = System.get_env("WHITELISTED_DOMAINS")

    allowed_domains =
      if is_nil(whitelisted_domains),
        do: ["localhost"],
        else: String.split(whitelisted_domains, ",")

    Enum.member?(allowed_domains, conn.host)
  end
end

and here is my endpoint.ex

Plug DbserviceWeb.DomainWhitelistPlug

9

From the docs on Plug.Conn.send_resp:

Note that this function does not halt the connection, so if subsequent plugs try to send another response, it will error out. Use halt/1 after this function if you want to halt the plug pipeline.

The “it will error out” is referring to AlreadySentError, as you’ve encountered.

10

Agreed with @al2o3cr that’s the likely culprit. If you verify that send_resp/2 is unexpectedly being called when running your tests, then you’d want to double check your allowed_domained?/1 function since it’d likely be returning false when you expect it to return true in your tests.

11

@al2o3cr but after Implementing the above solution
now here is my domain_whitelist_plug.ex

defmodule DbserviceWeb.DomainWhitelistPlug do
  import Plug.Conn

  def init(options) do
    options
  end

  def call(conn, _options) do
    if allowed_domains?(conn) do
      conn
    else
      conn
      |> send_resp(403, "Not Authorized")
      |> halt()
    end
  end

  defp allowed_domains?(conn) do
    whitelisted_domains = System.get_env("WHITELISTED_DOMAINS")

    allowed_domains =
      if is_nil(whitelisted_domains),
        do: ["localhost"],
        else: String.split(whitelisted_domains, ",")

    Enum.member?(allowed_domains, conn.host)
  end
end

now when I runs my test I encountered this below error

test index lists all user (DbserviceWeb.UserControllerTest)
     test/dbservice_web/controllers/user_controller_test.exs:71
     ** (RuntimeError) expected response with status 200, got: 403, with body: 
     "Not Authorized"
     code: [head | _tail] = json_response(conn, 200)
     stacktrace:
       (phoenix 1.6.10) lib/phoenix/test/conn_test.ex:369: Phoenix.ConnTest.response/2
       (phoenix 1.6.10) lib/phoenix/test/conn_test.ex:415: Phoenix.ConnTest.json_response/2
       test/dbservice_web/controllers/user_controller_test.exs:73: (test)
12

Like I mentioned above,

Try throwing a require IEx; IEx.pry or an IO.inspect(conn.host); IO.inspect(allowed_domains) in your allowed_domains?/1 function.

At the moment your test expects Enum.member?(allowed_domains, conn.host) to return true, but it’s instead returning false which leads to the error you’re seeing.

13

@codeanpeace

  defp allowed_domains?(conn) do
    whitelisted_domains = System.get_env("WHITELISTED_DOMAINS")

    allowed_domains =
      if is_nil(whitelisted_domains),
        do: ["localhost"],
        else: String.split(whitelisted_domains, ",")

    IO.inspect(conn.host)
    IO.inspect(allowed_domains)

    Enum.member?(allowed_domains, conn.host)
  end
end

now when I runs I got this

"www.example.com"
["localhost"]


  1) test index lists all user (DbserviceWeb.UserControllerTest)
     test/dbservice_web/controllers/user_controller_test.exs:71
     ** (RuntimeError) expected response with status 200, got: 403, with body: 
     "Not Authorized"
     code: [head | _tail] = json_response(conn, 200)
     stacktrace:
       (phoenix 1.6.10) lib/phoenix/test/conn_test.ex:369: Phoenix.ConnTest.response/2
       (phoenix 1.6.10) lib/phoenix/test/conn_test.ex:415: Phoenix.ConnTest.json_response/2
       test/dbservice_web/controllers/user_controller_test.exs:73: (test)      


Finished in 0.2 seconds (0.00s async, 0.2s sync)
1 test, 1 failure
14

Yeah, that’s what I had suspected. In your testing environment, your host is "www.example.com" which is not one of your allowed_domains?. There’s a few ways of addressing:

  • set it in your testing environment under the environment variable "WHITELISTED_DOMAINS"
  • add it as another default when env var is not set e.g. ["localhost", "www.example.com"]
  • skip the check entirely when Mix.env() == :test

If it were me, I’d probably set up the environments corrently and then remove the allowed_domains = ... entirely while using Enum.member?(whitelisted_domains, conn.host) instead.

15

@codeanpeace Thanks
you mean like this right ?


  defp allowed_domains?(conn) do
    whitelisted_domains = System.get_env("WHITELISTED_DOMAINS")

    allowed_domains =
      if is_nil(whitelisted_domains),
        do: ["localhost", "www.example.com"],
        else: String.split(whitelisted_domains, ",")

    if Mix.env() == :test do
      true
    else
      Enum.member?(allowed_domains, conn.host)
    end
  end
end
16

Not entirely…

You dont need both ["localhost", "www.example.com"], and if Mix.env() == :test ... changes. Just one of those changes should be enough.

When I wrote the above, I meant this setting up your environments such that System.get_env("WHITELISTED_DOMAINS") returns ["localhost"] when Mix.env() == :dev and ["www.example.com"] when Mix.env() == :test, which would allow for the refactor below.

  defp allowed_domains?(conn) do
    Enum.member?(System.get_env("WHITELISTED_DOMAINS"), conn.host)
  end

Note that there is System.put_env/2 function that could be useful for this approach.

1 Like
17

@codeanpeace Thanks , I think it solved the above issue but now I get another issue related to empty response.
now here is my domain_whitelist_plug.ex

defmodule DbserviceWeb.DomainWhitelistPlug do
  import Plug.Conn

  def init(options) do
    options
  end

  def call(conn, _options) do
    if allowed_domains?(conn) do
      conn
    else
      conn
      |> send_resp(403, "Not Authorized")
      |> halt()
    end
  end

  defp allowed_domains?(conn) do
    whitelisted_domains = System.get_env("WHITELISTED_DOMAINS")

    allowed_domains =
      if is_nil(whitelisted_domains),
        do: ["localhost"],
        else: String.split(whitelisted_domains, ",")

    if Mix.env() == :test do
      true
    else
      Enum.member?(allowed_domains, conn.host)
    end
  end
end

but when I run the test code provided above , it give me this error

 test index lists all user (DbserviceWeb.UserControllerTest)
     test/dbservice_web/controllers/user_controller_test.exs:71
     ** (MatchError) no match of right hand side value: []
     code: [head | _tail] = json_response(conn, 200)
     stacktrace:
       test/dbservice_web/controllers/user_controller_test.exs:73: (test)

but when I hit the api endpoint url http://localhost:4000/api/user I get the data,
but when I run the test here I get empty response in the response body

here is my IO.inspect(conn)

%Plug.Conn{
  adapter: {Plug.Adapters.Test.Conn, :...},
  assigns: %{layout: false, user: []},
  body_params: %{},
  cookies: %{},
  halted: false,
  host: "www.example.com",
  method: "GET",
  owner: #PID<0.454.0>,
  params: %{},
  path_info: ["api", "user"],
  path_params: %{},
  port: 80,
  private: %{
    DbserviceWeb.Router => {[], %{PhoenixSwagger.Plug.SwaggerUI => []}},
    :before_send => [#Function<0.11807388/1 in Plug.Telemetry.call/2>],
    :phoenix_action => :index,
    :phoenix_controller => DbserviceWeb.UserController,
    :phoenix_endpoint => DbserviceWeb.Endpoint,
    :phoenix_format => "json",
    :phoenix_layout => {DbserviceWeb.LayoutView, :app},
    :phoenix_recycled => false,
    :phoenix_request_logger => {"request_logger", "request_logger"},
    :phoenix_router => DbserviceWeb.Router,
    :phoenix_template => "index.json",
    :phoenix_view => DbserviceWeb.UserView,
    :plug_session_fetch => #Function<1.84243074/1 in Plug.Session.fetch_session/1>,
    :plug_skip_csrf_protection => true
  },
  query_params: %{},
  query_string: "",
  remote_ip: {127, 0, 0, 1},
  req_cookies: %{},
  req_headers: [{"accept", "application/json"}],
  request_path: "/api/user",
  resp_body: "[]",
  resp_cookies: %{},
  resp_headers: [
    {"content-type", "application/json; charset=utf-8"},
    {"cache-control", "max-age=0, private, must-revalidate"},
    {"x-request-id", "F1NFjCHCBySi7EcAAAOB"}
  ],
  scheme: :http,
  script_name: [],
  secret_key_base: :...,
  state: :sent,
  status: 200
}

here is my usercontoller

  def index(conn, params) do
    param = Enum.map(params, fn {key, value} -> {String.to_existing_atom(key), value} end)
    user =
      Enum.reduce(param, User, fn
        {key, value}, query ->
          from u in query, where: field(u, ^key) == ^value

          _, query ->
            query
          end)
          |> Repo.all()
          IO.inspect(user)
    render(conn, "index.json", user: user)
  end
18

Have you gone through the Introduction to Testing guide for Phoenix? They’re quite detailed and well worth the read.

My first thought would be that there aren’t any users in your test database since I don’t see any setup. When you manually hit the endpoing, you’re seeing the results from the dev database. The test database is a blank slate by design/default and gets reset between each test.

For example, see how the guide’s example for testing the index action of JSON controllers matches against an empty list and compare it to how the example for testing the delete action creates an article as part of its setup.

19

@codeanpeace Thanks, It’s very helpful .
One last thing I want to ask that , after pushing the code for the domain_whitelist_plug.ex file, my checks on GitHub failed with the error message Function Mix.env/0 does not exist." . This error is likely due to some issue with the Mix.env function. can you help me on this ?

20