Different error on dev and production

phoenix
errors
file-uploading
#1

Hey there,
When I upload a file that is too big, in dev I am getting the phoenix error that looks like this:

Plug.Parsers.RequestTooLargeError at POST/topics/4gvJMZkdbBpN0e65/post_attachment

the request is too large. If you are willing to process larger requests, please give a :length to Plug.Parsers

and has a status of: 413 Request Entity Too Large

but when I deploy the app this error gets swallowed it seems, and just an empty 500 is returned.
in the AWS logs I can see the 413.

How to handle this error to have the same error the same way on both environments?

0 Likes

#2

First step is probably to try running it in “prod mode” locally, see if you can reproduce the issue. But after that it depends on what your production environment looks like. You’re on AWS? So you have an ELB in front of your app? Are you also running a reverse proxy like nginx?

0 Likes

#3

Thanks for the answer, I am able to reproduce it locally, I am not convinced that the aws stuff is related.
Here it is:

** (exit) an exception was raised:
    ** (Protocol.UndefinedError) protocol Phoenix.HTML.Safe not implemented for %{data: nil, errors: nil, meta: %{message: "Something went wrong.", status: 500}}. This protocol is implemented for: Decimal, Integer, Float, Tuple, Phoenix.HTML.Form, Date, BitString, DateTime, NaiveDateTime, List, Time, Atom
        (phoenix_html) deps/phoenix_html/lib/phoenix_html/safe.ex:1: Phoenix.HTML.Safe.impl_for!/1
        (phoenix_html) deps/phoenix_html/lib/phoenix_html/safe.ex:15: Phoenix.HTML.Safe.to_iodata/1
        (phoenix) lib/phoenix/controller.ex:729: Phoenix.Controller.__put_render__/5
        (xd_endpoint) lib/xd_endpoint/endpoint.ex:1: XD.Endpoint.instrument/4
        (phoenix) lib/phoenix/endpoint/render_errors.ex:75: Phoenix.Endpoint.RenderErrors.instrument_render_and_send/5
        (phoenix) lib/phoenix/endpoint/render_errors.ex:62: Phoenix.Endpoint.RenderErrors.__catch__/5
        (plug_cowboy) lib/plug/cowboy/handler.ex:18: Plug.Adapters.Cowboy.Handler.upgrade/4
        (cowboy) /application/xd_umbrella/deps/cowboy/src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

the same thing happens btw if i don’t change the lenght value in Plug.Parsers

0 Likes

#4

so it seems lit wants to give back an html error and not a json

0 Likes

#5

So it only happens when you run with MIX_ENV=prod? I’d compare the dev and prod configs, maybe take a look at render_errors and debug_errors, off the top of my head.

1 Like

#6

this is how the assings look
I am trying to upload an file in form data

%{
  conn: %Plug.Conn{
    adapter: {Plug.Adapters.Cowboy.Conn, :...},
    assigns: %{
      kind: :error,
      layout: false,
      reason: %Plug.Parsers.RequestTooLargeError{
        message: "the request is too large. If you are willing to process larger requests, please give a :length to Plug.Parsers",
        plug_status: 413
      },
      stack: [
        {Plug.Parsers, :reduce, 7, [file: 'lib/plug/parsers.ex', line: 279]},
        {DotooEndpoint.Endpoint, :plug_builder_call, 2,
         [file: 'lib/xd_endpoint/endpoint.ex', line: 1]},
        {DotooEndpoint.Endpoint, :call, 2,
         [file: 'lib/xd_endpoint/endpoint.ex', line: 1]},
        {Plug.Adapters.Cowboy.Handler, :upgrade, 4,
         [file: 'lib/plug/cowboy/handler.ex', line: 18]},
        {:cowboy_protocol, :execute, 4,
         [
           file: '/application/xd_umbrella/deps/cowboy/src/cowboy_protocol.erl',
           line: 442
         ]}
      ]
    },
    before_send: [],
    body_params: %Plug.Conn.Unfetched{aspect: :body_params},
    cookies: %Plug.Conn.Unfetched{aspect: :cookies},
    halted: false,
    host: "localhost",
    method: "POST",
    owner: #PID<0.595.0>,
    params: %{},
    path_info: ["topics", "4gvJMZkdbBpN0e65", "post_attachment"],
    path_params: %{},
    port: 4000,
    private: %{
      phoenix_endpoint: XD.Endpoint,
      phoenix_format: "html",
      phoenix_layout: false,
      phoenix_template: "413.html",
      phoenix_view: XD.ErrorView
    },
    query_params: %{},
    query_string: "",
    remote_ip: {0, 0, 0, 0, 0, 65535, 44051, 1},
    req_cookies: %Plug.Conn.Unfetched{aspect: :cookies},
    req_headers: [
      {"authorization",
       "Bearer xxx"},
      {"content-type",
       "multipart/form-data; boundary=--------------------------773415269124190613492413"},
      {"cache-control", "no-cache"},
      {"postman-token", "88c8a61c-ed3e-4002-925e-e0caf67e338f"},
      {"user-agent", "PostmanRuntime/7.6.0"},
      {"accept", "*/*"},
      {"host", "localhost:4000"},
      {"accept-encoding", "gzip, deflate"},
      {"content-length", "31457602"},
      {"connection", "keep-alive"}
    ],
    request_path: "/topics/4gvJMZkdbBpN0e65/post_attachment",
    resp_body: nil,
    resp_cookies: %{},
    resp_headers: [{"cache-control", "max-age=0, private, must-revalidate"}],
    scheme: :http,
    script_name: [],
    secret_key_base: :...,
    state: :unset,
    status: 413
  },
  kind: :error,
  reason: %Plug.Parsers.RequestTooLargeError{
    message: "the request is too large. If you are willing to process larger requests, please give a :length to Plug.Parsers",
    plug_status: 413
  },
  stack: [
    {Plug.Parsers, :reduce, 7, [file: 'lib/plug/parsers.ex', line: 279]},
    {XDEndpoint.Endpoint, :plug_builder_call, 2,
     [file: 'lib/xd_endpoint/endpoint.ex', line: 1]},
    {XDEndpoint.Endpoint, :call, 2,
     [file: 'lib/xd_endpoint/endpoint.ex', line: 1]},
    {Plug.Adapters.Cowboy.Handler, :upgrade, 4,
     [file: 'lib/plug/cowboy/handler.ex', line: 18]},
    {:cowboy_protocol, :execute, 4,
     [
       file: '/application/xd_umbrella/deps/cowboy/src/cowboy_protocol.erl',
       line: 442
     ]}
  ],
  view_module: XDEndpoint.ErrorView,
  view_template: "413.html"
}

0 Likes

#7

yes it only happens in production there it gives back the html error altough i’d prefer the json in both cases

0 Likes

#8

I was able to make it arrive here:

  def render("413.json", _params) do
    IO.puts('here')

    %{
      meta: %{status: 413, message: "Too large file"},
      errors: nil,
      data: nil
    }
  end

but at the end it still put out the empty 500 error

and it logs out this:

 (Protocol.UndefinedError) protocol Phoenix.HTML.Safe not implemented for %{data: nil, errors: nil, meta: %{message: "Too large file", status: 413}}.

the same as with the 500 earlier

0 Likes

#9

You are returning a structure from the render function instead of a binary. If you want it to return json then you need to convert it to json instead of an elixir map. :slight_smile:

  def render("413.json", _params) do
    IO.puts('here')

    %{
      meta: %{status: 413, message: "Too large file"},
      errors: nil,
      data: nil
    } |> Jason.encode!()
  end

Other than that you shouldn’t need to return any body for a 413 as the status code itself already told the issue, that’s not the body’s job.

1 Like

#10

That can’t be right, I’m returning maps from my render functions and I get JSON. I couldn’t tell you why though, is it inferred from the view name?

Also, the post author says it works with MIX_ENV=dev, it just fails with prod, which made me suggest it is config related.

Edit: Googled a bit and the Phoenix View documentation shows rendering JSON from a map in an example. So it should work.

https://hexdocs.pm/phoenix/Phoenix.View.html#module-rendering

1 Like

#11

Yeah, I ended up doing this, and also the guy who wanted it figured it out that the status code is enough for him too… Thank you for the answer

0 Likes

#12

Not the view name but rather what ‘higher up’ in the callstack is calling your render function. In a base error rendrerer there’s very little above it and I’m not sure if the content-type requested by the client is followed or not (since it’s an error and the body shouldn’t be used for data anyway).

You can always return the stacktrace in that function and see what it’s going through, there might be an option? :slight_smile:

Edit: Googled a bit and the Phoenix View documentation shows rendering JSON from a map in an example. So it should work.

That’s for rendering from controllers, not errors though, I’m unsure about error handling.

2 Likes

#13

Yes, the problem was that the production didn’t return the correct code either

1 Like

#14

I’m curious what the content-type is of the response then.

@OvermindDL1 as far as I can tell, the error view is set as a config option on the endpoint (render_errors) where you can specify a list of content types. Generating a fresh phoenix app with no HTML gives you an error view that has this comment

  # If you want to customize a particular status code
  # for a certain format, you may uncomment below.
  # def render("500.json", _assigns) do
  #   %{errors: %{detail: "Internal Server Error"}}
  # end

so I see no reason why returning maps wouldn’t work here either.

In fact, @benonymus, I’m curious what your render_errors config option looks like in config/config.exs, config/dev.exs and config/prod.exs. Maybe you’ve put JSON in one and HTML in another? It should look something like this

render_errors: [view: HelloWeb.ErrorView, accepts: ~w(json)],
1 Like

#15

Hmm, good question then. Is there a test for this in the phoenix testsuite?

0 Likes

#16

In fact, I can reproduce @benonymus’s error by specifying html in my render_errors.

So in my config/config.exs I replaced

render_errors: [view: HelloWeb.ErrorView, accepts: ~w(json)],

with

render_errors: [view: HelloWeb.ErrorView, accepts: ~w(html)],

I get

[error] #PID<0.603.0> running HelloWeb.Endpoint (connection #PID<0.602.0>, stream id 1) terminated
Server: localhost:4000 (http)
Request: POST /nowhere
** (exit) an exception was raised:
    ** (UndefinedFunctionError) function Phoenix.HTML.Safe.to_iodata/1 is undefined (module Phoenix.HTML.Safe is not available)
        Phoenix.HTML.Safe.to_iodata(%{errors: %{detail: "Not Found"}})
        (phoenix) lib/phoenix/controller.ex:729: Phoenix.Controller.__put_render__/5
        (hello) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/endpoint/render_errors.ex:75: Phoenix.Endpoint.RenderErrors.instrument_render_and_send/5
        (phoenix) lib/phoenix/endpoint/render_errors.ex:62: Phoenix.Endpoint.RenderErrors.__catch__/5
        (phoenix) lib/phoenix/endpoint/cowboy2_handler.ex:33: Phoenix.Endpoint.Cowboy2Handler.init/2
        (cowboy) /Users/johanna/workspace/hello/deps/cowboy/src/cowboy_handler.erl:41: :cowboy_handler.execute/2
        (cowboy) /Users/johanna/workspace/hello/deps/cowboy/src/cowboy_stream_h.erl:296: :cowboy_stream_h.execute/3
        (cowboy) /Users/johanna/workspace/hello/deps/cowboy/src/cowboy_stream_h.erl:274: :cowboy_stream_h.request_process/3
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
1 Like

#17

Hmmm, I wonder if that error message could be improved to guide users to the correct fix for their needs. At the very least I believe we can say that when only html is specified in render_errors that only an iolist or binary are valid inputs and other inputs should result in an error message.

2 Likes

#18

render_errors: [view: DotooEndpoint.ErrorView, accepts: ~w(html json)]

I have both, this is an umbrella application and we have a endpoint project that acts as a gateway.
also as you can see from my earlier comment

it is by default looking for 413.html but until now i only had .json error_views.

and even if I pass the json error to it manually I am getting the same error as you reproduced

the difference that I can see between the dev and production envs is that the dev has debug_errors shown while the production doesn’t

also even if i return json the reponse still stays html, postman cannot beautify it

0 Likes