(Bandit.HTTPError) Header read socket error: :closed

Phoenix Forum Questions / Help
,
1

I am playing around a purely LiveView based UI for upcoming project. It is basically a fresh Phoenix application with a few function components and a LiveView using them. Things seem to work well. Yet on occasions I keep getting a dreaded red line in the log:

(Bandit.HTTPError) Header read socket error: :closed

So far I couldn’t figure out any pattern when it pops up or what could be triggering it.

LiveView version is “1.0.0-rc.6”, Bandit is at “1.5.7”.

Any clues?

1 Like
2

Having the same issue like you. And the related deps are the same ones. Investigated the reason but found no clue.

(Posting this reply is to convey that the issue you encountered is not unique.

2 Likes
3

Thank you - it’s good to know that this may not be something very specific to my setup only.

4

Getting the same with stacktrace:

(bandit)
Bandit.HTTPTransport.Bandit.HTTP1.Socket.request_error!/2
lib/bandit/http1/socket.ex:420

(bandit)
Bandit.HTTPTransport.Bandit.HTTP1.Socket.send_data/3
lib/bandit/http1/socket.ex:357

(bandit)
Bandit.Adapter.send_data/3
lib/bandit/adapter.ex:242

(bandit)
Bandit.Adapter.send_resp/4
lib/bandit/adapter.ex:140

(plug)
Plug.Conn.send_resp/1
lib/plug/conn.ex:444

(plug)
Plug.Debugger.__catch__/5
lib/plug/debugger.ex:162
5

Isn’t it possible to run bandit in verbose mode? That might shine some light on what is actually happening under the hood.

6

Running into this one as well. It started with a recent release and no other changes. Maybe we should just create a ticket?

1 Like
7

I get the same with live_view 0.20.17, bandit 1.5.7.
And occasionally

[error] ** (Bandit.HTTPError) Header read timeout
1 Like
8

Found related issues:

And, I’d like to quote words from @mtrudel.

Where does this error come from?

This is a wholly normal error; one that comes from the client closing the connection before the server has fully sent its response.

Can i ignore this error?

Unless you control the client, you can safely ignore these. They correspond to client errors, disconnections, and the like.

Which configuration can i use to control this behaviour?

It is configurable within Bandit via the http_errors.log_protocol_errors configuration point

I’ll try the mentioned configuration and report the result at here later.

Edit

In short, it works.

1. Reproduce the error

As mtrudel said, The error (Bandit.HTTPError) Header read socket error: :closed comes from a client-side issue. But, it’s hard to reproduce it using command like curl. Therefore, I attempted another method to simulate a client-side issue.

# build a malformed request
$ curl 127.0.0.1:4000 -H 'Host:'

Within a project generated by mix phx.new --adapter bandit ..., it will report an error:

[error] ** (Bandit.HTTPError) Unable to obtain host and port: No host header

2. Set the option

After setting log_protocol_errors: false:

config :demo, DemoWeb.Endpoint,
  http: [
    http_options: [log_protocol_errors: false],
    # ...
  ],
  # ...

3. Result

The error won’t be reported anymore.

4. (extra) How does Cowboy adapter handle malformed requests?

I also generated a project using cowboy adapter.

When I sent a malformed request to the server, the server doesn’t report anything.

5. (extra) Investigate ‘Is it safe to suppress this kind of error’?

I noticed the changelog said:

Add log_protocol_errors option to optionally quell console logging of 4xx errors generated by Bandit. Defaults to true for now; may switch to false in the future based on adoption

As it said that log_protocol_errors is for suppressing 4xx errors - which are client errors, IMO, it’s safe to do that.

2 Likes
9

Yep, the log_protocol_errors option does just that; controls the reporting of protocol facing errors (as opposed to exceptions raised from within your Plug.call/2 implementation, not returning a Plug.Conn struct, etc). Its default value of true is mostly a historical artifact from Bandit’s early days, and will almost certainly start defaulting to false at some point in the future.

Cowboy doesn’t log anything in this case because their policy is mostly one of silence in the face of unexpected conditions. If you browse Bandit’s issue history, a substantial volume of reported issues are actually from users’ own code (or other libraries) that were always erroring under Cowboy but not being reported. These are no different.

2 Likes
10

I am not sure I understand the direction the discussion is heading to… for me suppressing information (logging) about invalid/error conditions is hardly going to fix the actual causes of those error conditions. I don’t think anyone is blaming “Bandit” for delivering this information. For me it’s more important that if this is a correct report of an error in the communication with Bandit then it’s great to know that something is not right. That something else is in fact misbehaving in some ways.

1 Like
11

While I agree that it’s great that Bandit actually covers those cases it’s also true that for many server use cases errors caused by misbehaving clients are completely non-actionable and therefore noise. Therefore having a flag to enable or disable such logging of errors is the way to go. People which happen to control clients (e.g. in embedded setups) can certainly appreciate errors around misbehaving clients.

2 Likes
12

ACK - sure thing. If I am in the position of a generic Bandit server admin, where I can’t do much about those errors, then this is good to have an option to reduce the noise.

My feeling is that this is not the actual issue at hand though. At least that wasn’t the reason for me to start this thread. The issue at hand is that there is a “fresh” Phoenix LiveView application (something I supposedly control), which in some way misbehaves, and I find it good that Bandit reports that. If we have an answer from “Bandit people” that this is nothing on their side it’s also good. But then the real question is what part of such application misbehaves and how to correct it.

1 Like
13

Actually, Bandit.HTTPErrors are actually very descriptive. I gave it a search across the repo, and got:

(Bandit.HTTPError) Header read HTTP error
(Bandit.HTTPError) Invalid HTTP version: {0, 9}
(Bandit.HTTPError) Unable to obtain host and port: No host header
(Bandit.HTTPError) Header contains invalid port
(Bandit.HTTPError) Unsupported request target (RFC9112§3.2)
(Bandit.HTTPError) schemeURI is not supported
(Bandit.HTTPError) Request URI is too long
(Bandit.HTTPError) Header too long
(Bandit.HTTPError) Too many headers
(Bandit.HTTPError) Content length unknown error: \"invalid content-length header (RFC9112§6.3.5)
(Bandit.HTTPError) Body read timeout
(Bandit.HTTPError) Excess body read
(Bandit.HTTPError) HTTP method POST unsupported
(Bandit.HTTPError) 'upgrade' header must contain 'websocket'
(Bandit.HTTPError) 'connection' header must contain 'upgrade'
(Bandit.HTTPError) 'sec-websocket-key' header is absent
(Bandit.HTTPError) 'sec-websocket-version' header must equal '13'
(Bandit.HTTPError) Header read HTTP error: \"GARBAGE\\r\\n\"
(Bandit.HTTPError) Header read timeout
(Bandit.HTTPError) closed

As you can see, if we encounter an HTTP error caused by the client, Bandit is generally able to describe the reason well.

It’s just that some reasons are not so intuitive to interpret. For example, the one we initially mentioned - (Bandit.HTTPError) Header read socket error: :closed, which means “the client closing the connection before the server has fully sent its response”.

So, through the HTTPError, we can know which part (at least the client part) of HTTP communication has an issue. As for how to correct it, perhaps it involves ensuring that both the client and server adhere to the HTTP standards (I don’t have much experience in this area).

1 Like
14

Thank you for elaborating on this, just – once more – this happens (to me) on a “fresh” Phoenix/LiveView application. From what I read here, the same (or other but somewhat similar?) happens to other people too. Which – if we exclude “Bandit”'s fault – seems to suggest that current LiveView implementation has some problems, doesn’t it?

15

Everything technical that folks have said so far is correct to my understanding; the issue (as you rightly point out) is that clients sometimes close connections (either explicitly or not) at unexpected times. This is the reality of network facing services and trying to fix it in the large is a a fool’s errand.

That being said, if you’re seeing this in a new project, running on localhost, in an up to date browser, then there’s less of an expectation of this sort of thing. Client closure errors are the same at every scope (that is, we obviously don’t treat new / local / simple apps any different than we do the largest and most complex deployment), so the same analysis should hold. It may be worth looking into what is happening on the clients in this case, but I’d strongly suspect it has less to do with phoenix.js et al and more to do with how browsers implement their network stacks.