Phoenix fails in live_reloader when response is encoded with Msgpax

Phoenix Forum Questions / Help
1

I’ve made a PR #87 to add custom serializers for absinthe_plug, and I’ve also set up a repo idi-ot/absinthe_test to try it out with msgpax, but the query

curl -XPOST \
  -H "Content-Type:application/graphql" \
  -d "{posts {id}}" http://localhost:4000/api

fails with

[info] POST /api
[debug] ABSINTHE schema=nil variables=%{}
---
{posts {id}}
---
[info] Sent 200 in 1ms
[info] Sent 500 in 48ms
[error] #PID<0.432.0> running AbsintheTest.Web.Endpoint terminated
Server: localhost:4000 (http)
Request: POST /api
** (exit) an exception was raised:
    ** (UnicodeConversionError) invalid encoding starting at <<220, 0, 100>>
        (elixir) lib/list.ex:727: List.to_string/1
        (phoenix_live_reload) lib/phoenix_live_reload/live_reloader.ex:94: anonymous fn/2 in Phoenix.LiveReloader.before_send_inject_reloader/2
        (elixir) lib/enum.ex:1755: Enum."-reduce/3-lists^foldl/2-0-"/3
        (plug) lib/plug/conn.ex:971: Plug.Conn.run_before_send/2
        (plug) lib/plug/conn.ex:392: Plug.Conn.send_resp/1
        (phoenix) lib/phoenix/router/route.ex:161: Phoenix.Router.Route.forward/4
        (phoenix) lib/phoenix/router.ex:278: Phoenix.Router.__call__/1
        (absinthe_test) lib/absinthe_test/web/endpoint.ex:1: AbsintheTest.Web.Endpoint.plug_builder_call/2
        (absinthe_test) lib/plug/debugger.ex:123: AbsintheTest.Web.Endpoint."call (overridable 3)"/2
        (absinthe_test) lib/absinthe_test/web/endpoint.ex:1: AbsintheTest.Web.Endpoint.call/2
        (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
        (cowboy) /Users/asd/Developer/elixir/absinthe_test/deps/cowboy/src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

on the phoenix’s side.

Seems like phoenix tries to do something with the response (encoded by msgpax) and fails.

If I disable code_reloader, it works fine.

2

def encode!(data, _opts), do: Msgpax.pack!(data, iodata: false)

3

Thank you. It does work, but I would still want to use iodata.

4

with the code_reloader everything goes through:

  defp before_send_inject_reloader(conn, endpoint) do
    register_before_send conn, fn conn ->
      resp_body = to_string(conn.resp_body)

msgpack:

iex> (for id <- 1..16 do %{id: id, title: "post-#{id}", body: "body-#{id}"} end)|> Msgpax.pack!
[<<220, 0, 16>>,
 [131, [[164 | "body"], 166 | "body-1"], [[162 | "id"], 1]....

to_string breaks with

iex> (for id <- 1..16 do %{id: id, title: "post-#{id}", body: "body-#{id}"} end)|> Msgpax.pack! |> to_string
** (UnicodeConversionError) invalid encoding starting at <<220, 0, 16>>

works fine with 1…15:

iex> (for id <- 1..15 do %{id: id, title: "post-#{id}", body: "body-#{id}"} end)|> Msgpax.pack!             
[159,
 [131, [[164 | "body"], 166 | "body-1"], [[162 | "id"], 1]....

to_string

iex> (for id <- 1..15 do %{id: id, title: "post-#{id}", body: "body-#{id}"} end)|> Msgpax.pack! |> to_string
<<194, 159, 194, 131, 194, 164, 98, 111, 100, 121, 194, 166, 98, 111, 100, 121....

I’m not the one to say if the error is in msgpax or the code_reloader or iodata is not to be used - but obviously to_string is not happy about the <<220, 0, 16>> part;-)

With my limited knowledge iodata should be preferred yes.

1 Like
5

I’ll open an issue in phoenix_live_reloader.

1 Like
6

https://github.com/phoenixframework/phoenix_live_reload/issues/55

7

Maybe you can add your post there?

8

https://github.com/elixir-lang/elixir/issues/6186

9

Checking if resp_body is a list and then encoding it with IO.iodata_to_binary/1 seems to solve the problem somewhat.

defp before_send_inject_reloader(%{resp_body: resp_body} = conn, endpoint) do
  register_before_send conn, fn conn ->
    resp_body = if is_list(resp_body) do
      IO.iodata_to_binary(resp_body)
    else
      to_string(resp_body)
    end
    ...

Hope it will get merged #56.

10

Hmm, I would worry that messes with something and slows things down - eg this code keeps the “happy path” intact:
(resp_body is only used to check if it’s a html page and inject the code_reloader js in that case)

  defp before_send_inject_reloader(conn, endpoint) do
    register_before_send conn, fn conn ->
      resp_body = try do
        to_string(conn.resp_body)
      rescue
        _ -> false
      end

      if resp_body && inject?(conn, resp_body) && :code.is_loaded(endpoint) do
1 Like
11

Yeah, I thought about wrapping it in a try block, but then changed my mind, but it’s probably the right way to go.

12

Still, I wouldn’t worry about speed in dev environment.