How to debug a long error stack?

Questions / Help
, , , ,
1

How to debug, in general, an error stack like this one (bellow) ?
It’s mixed with errors coming from the Erlang dependencies. Notice it doesn’t point to a certain line of my (Elixir) code.

Where to start from, what is the error source, error initiator, errors flow? How to read it, from bottom to top or top to bottom?

[error] Ranch listener "qwerty" had connection process started with :cowboy_clear:start_link/4 at #PID<0.489.0> exit with reason:
 {
    :badarg,
    [
        {
            :erlang,
            :iolist_size,
            [
                {
                    :EXIT, #PID<0.487.0>,
                    :shutdown
                }
            ],
            [error_info: %{module: :erl_erts_errors
                }
            ]
        },
        {
            :cow_ws,
            :payload_length,
            1,
            [file: '.../deps/cowlib/src/cow_ws.erl', line: 725
            ]
        },
        {
            :cow_ws,
            :frame,
            2,
            [file: '.../deps/cowlib/src/cow_ws.erl', line: 666
            ]
        },
        {
            :cowboy_websocket,
            :websocket_send,
            2,
            [file: '.../deps/cowboy/src/cowboy_websocket.erl', line: 626
            ]
        },
        {
            :cowboy_websocket,
            :handler_call,
            6,
            [file: '.../deps/cowboy/src/cowboy_websocket.erl', line: 542
            ]
        },
        {
            :cowboy_http,
            :loop,
            1,
            [file: '.../deps/cowboy/src/cowboy_http.erl', line: 257
            ]
        },
        {
            :proc_lib,
            :init_p_do_apply,
            3,
            [file: 'proc_lib.erl', line: 226
            ]
        }
    ]
}

Thanks! :smiley:

2

I would start from top to bottom as this is a stack trace - look at what the exact error is and then check what data is being passing to library calls.

Error at this line (most likely as I don’t know your library versions) - matches your stack trace:

It has something to do with Payload length or indirectly Payload itself?

Little more context about the app would help - is this phoenix app or elixir app with cowboy web sockets ? What versions of libraries you are using?

2 Likes
3

Start from the top - this is the root cause of the error - :erlang.iolist_size/1 is raising :badarg. The most common cause of this is passing something in that’s not the right shape.

The next two frames for :cow_ws.payload_length and :cow_ws.frame aren’t particularly interesting.

The frame for :cow_ws.websocket_send gives us another piece of information: we’re trying to send whatever the badly-shaped data is down a websocket

But the really important one is the frame after that: :cow_ws.handler_call. This is the “brains” of the websocket handler:

This is why the stack doesn’t have any application code in it - the application code is called in the first part of handler_call, and then the result of that is what’s being handled when the crash happens.

Check your code that’s sending data via websockets; has a stray {:ok, "data that should be sent"} snuck in someplace where you’re expecting just "data that should be sent"?

2 Likes
4

Thanks a lot @al2o3cr and @kartheek. Now is clear for me… somewhat.

Little more context about the app would help - is this phoenix app or elixir app with cowboy web sockets ? What versions of libraries you are using?

No Phoenix, just Plug.Cowboy. v.2.5.2

Check your code that’s sending data via websockets; has a stray {:ok, "data that should be sent"} snuck in someplace where you’re expecting just "data that should be sent" ?

The payload is {:ok, obj}, where obj is a json. Then I still can’t get it why this error is raised.

This is just one example of this kind of errors. I’ve encountered many as such and didn’t know where to start from. I was just looking for a rationale/pattern for analyzing them.
That’s why I was asking initially how to debug these “in general” and not particularly this error above.

An observation: it’s very frustrating that most of these Elixir error stacks go deep to the underlying Erlang libraries. As my Erlang knowledge being nil, it’s almost impossible for me to understand and debug them. :pensive:

5

cowboy_websocket expects frames to match the cow_ws:frame type:

frame() :: {text, iodata()}
    | {binary, iodata()}
    | ping | {ping, iodata()}
    | pong | {pong, iodata()}
    | close | {close, iodata()} | {close, close_code(), iodata()}

close_code() :: 1000..1003 | 1006..1011 | 3000..4999

{:ok, "{\"some\":\"json data\"}"} is none of those, and it’s not an iodata either - so an error occurs.

I can’t offer you any shortcut - if you’re going to write code that interfaces with Erlang libraries (like cowboy_websocket) you’re going to need to learn some Erlang.

2 Likes
6

Finally solved this one, I was mistakenly sending the websocket’s state instead of the json serialized payload. Now it’s fine, thanks again @al2o3cr. However, this error stack so cryptic and unfriendly for (just) Elixir coder. :-1:t4:
So, you’re right, I need to get my hands dirty with Erlang. In this case I need to understand the underlying implementation of this bloody cowboy_websocket. :face_with_raised_eyebrow: …iodata and such.
I consider a huge drawback for Elixir that is based so much on Erlang’s libraries. (immaturity ?). This remembers me Java in the late '90s. Or C#'s infancy.

7

It’s the other way round. Elixir’s biggest strength is that it is built on the solid foundation of Erlang, and it interops with it fairly seamlessly. I recommend you learn to love Erlang, as there is no escaping it :wink:

8

Love… I really don’t think so. :nauseated_face: It’s more about necessity. This reminds me of Tcl, had to learn it ages ago, to perform some tasks. I did it, but hated it. Even today I have bad dreams about it.
I really love Elixir instead. Too bad they are linked together at the Elixir use level. IMO Elixir should evolve into a complete abstraction from Erlang.

LE: Java lang is running on JVM, written in C/C++. One doesn’t have to know those in order to master Java. Right ?

9

The comparison is a bit off. It‘s more like Scala and Java. Both run on the JVM and can interop with each other. If you depend from a Scala project on a Java library you’re better off knowing Java as well.

I also found that erlang becomes easier and easier to read with getting to know elixir.

2 Likes
10

I mean learn to love it for your own satisfaction in life. It’s not going to split from Erlang and life is too sweet to worry about things out of your control.

And yes your analogy is off. You are largely freed from writing C by the Erlang team, who are writing it for you.

1 Like
11

My analogy might be off but I hope you got the idea.

You are largely freed from writing C by the Erlang team, who are writing it for you.

I would prefer to debug errors in C (a common language) rather than in Erlang. :stuck_out_tongue_winking_eye:

Anyway, let’s end this.
Thanks everybody,
Cheers.