HTTPoison Error: Error reason closed

Questions / Help
1

Has anyone come across the following response to HTTPoison before? The trace doesn’t seem to provide any clue as to the cause.

HTTPoison version “1.7.0”

:hackney_trace.enable(:max, :io)

response =   HTTPoison.get("https://www.mybramble.co.uk",headers) 
# {:error, %HTTPoison.Error{id: nil, reason: :closed}}

# Trace
hackney trace 80 <0.338.0> 2020:12:23 18:09:24 4629] request
   Content: [{module,hackney},
             {line,313},
             {method,get},
             {url,{hackney_url,hackney_ssl,https,<<"www.mybramble.co.uk">>,
                               <<>>,<<>>,<<>>,<<>>,"www.mybramble.co.uk",443,
                               <<>>,<<>>}},
             {headers,[{<<"Content-Type">>,
                        <<"application/x-www-form-urlencoded">>},
                       {<<"User-Agent">>,
                        <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36">>}]},
             {body,<<>>},
             {options,[]}]
[hackney trace 60 <0.338.0> 2020:12:23 18:09:24 4633] request without proxy
   Content: [{module,hackney},{line,685}]
[hackney trace 60 <0.338.0> 2020:12:23 18:09:24 4633] connect
   Content: [{module,hackney_connect},
             {line,32},
             {transport,hackney_ssl},
             {host,"www.mybramble.co.uk"},
             {port,443},
             {dynamic,true}]
[hackney trace 80 <0.377.0> 2020:12:23 18:09:24 4635] no socket in the pool
   Content: [{module,hackney_pool},{line,101},{pool,default}]
[hackney trace 80 <0.377.0> 2020:12:23 18:09:25 4144] connect error
   Content: [{module,hackney_pool},{line,121}]
[hackney trace 80 <0.338.0> 2020:12:23 18:09:25 4144] connect error
   Content: [{module,hackney_connect},{line,233}]
2

can you share a sample HTTP call with headers?

Also, thats not new, I had these such 1000+ errors in past.

If you are just starting a new app use Finch instead.

3

Here is the output of curl. It’s a 301 response:
curl -v https://www.mybramble.co.uk/

*   Trying 107.162.160.23...
* TCP_NODELAY set
* Connected to www.mybramble.co.uk (107.162.160.23) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: jurisdictionCountryName=GB; businessCategory=Private Organization; serialNumber=03974521; C=GB; L=Glasgow; O=Ecebs Limited; CN=mybramble.co.uk
*  start date: Oct 29 11:54:05 2019 GMT
*  expire date: Nov 11 15:52:39 2021 GMT
*  subjectAltName: host "www.mybramble.co.uk" matched cert's "www.mybramble.co.uk"
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=Starfield Technologies, Inc.; OU=http://certs.starfieldtech.com/repository/; CN=Starfield Secure Certificate Authority - G2
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.mybramble.co.uk
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Wed, 23 Dec 2020 21:25:54 GMT
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Location: https://www.mybramble.co.uk/RetailWeb/
< Content-Length: 246
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
< Via: 1.1 fra1-bit31
< Set-Cookie: TS0174d616=0126fcb6f0752d24ba90fb14c212472ca18f6edc753dbea0c1f785145c0eff9548c828dd9cf3aa238b324a64c99e48342d28e737ea; Path=/; Secure; HTTPOnly
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.mybramble.co.uk/RetailWeb/">here</a>.</p>
</body></html>
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):
4

Any luck here, I am also facing the same issue. Will it help to switch to some other library?

5

In the OP’s case: https://www.mybramble.co.uk the ssl handshake fails very early on. I doubt finch can work better because it must be using the same :ssl underneath. Can you pass ssl: [log_level: :all, verify: :verify_none] to HTTPosion to see if you really have the same issue?

6

Hey, I didn’t get to the bottom of this sadly. As for it being an ssl issue I didn’t see anything particularly odd with the ssl at the time and have not tried since. I ended up using Spring boot as the RestTemplate worked with no issues. This as well as it working with curl suggested that the issue is with something in the elixir implementation.

7

Have a go with Finch. I had another project which is using it and that url seems to work with it. This could be due to a change in the ssl config on the server side. Let me know how you get on and if you can share the url you are getting the error from.

8

After enabling all the logs, It is emitting so many logs including binary data and certificates which has no use to me. We are facing this issue while connecting to a third party service. This is an intermittent issue, we see this twice or thrice a day but subsequent requests work fine. I can not share the host details. It happens only in production so I don’t know how to reproduce this in our test environment. The curl details -

  • Trying XX.XXX.XX.XXX…
  • TCP_NODELAY set
  • Connected to {{host_name}} (XX.XXX.XX.XXX) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/cert.pem
    CApath: none
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
  • TLSv1.2 (IN), TLS handshake, Server hello (2):
  • TLSv1.2 (IN), TLS handshake, Certificate (11):
  • TLSv1.2 (IN), TLS handshake, Server key exchange (12):
  • TLSv1.2 (IN), TLS handshake, Server finished (14):
  • TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
  • TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
  • TLSv1.2 (OUT), TLS handshake, Finished (20):
  • TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
  • TLSv1.2 (IN), TLS handshake, Finished (20):
  • SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
  • ALPN, server accepted to use h2
  • Server certificate:
  • subject: C=US; ST={{some}}; L={{O}}; O={{o}}, Inc.; CN=*.{{some}}.com
  • start date: Jun 7 00:00:00 2021 GMT
  • expire date: Jul 8 23:59:59 2022 GMT
  • subjectAltName: host “{{hostname}}” matched cert’s “*.{{some}}.com”
  • issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
  • SSL certificate verify ok.
  • Using HTTP2, server supports multi-use
  • Connection state changed (HTTP/2 confirmed)
  • Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  • Using Stream ID: 1 (easy handle 0x7f8c20009600)

POST {{url_paath}} HTTP/2
Host: {{host_name}}
User-Agent: curl/7.64.1
Accept: /
Authorization: Basic {{creds}}
Content-Type: application/json
Content-Length: 105

  • Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
  • We are completely uploaded and fine
    < HTTP/2 201
    < date: Thu, 18 Nov 2021 17:03:43 GMT
    < content-type: application/json
    < server: nginx
    < x-content-type-options: nosniff
    < x-xss-protection: 1; mode=block
    < cache-control: no-cache, no-store, max-age=0, must-revalidate
    < pragma: no-cache
    < expires: 0
    < x-frame-options: DENY
    <
  • Connection #0 to host {{hostname}} left intact
    {{response}}}* Closing connection 0
9

Is it something to do with MAX_CONCURRENT_STREAMS ?

10

In my case, The server was closing the connection because it was not allowing multiple requests on the same connection. we were using the hackney connection pool to reuse the existing connection if any.
Now, we are creating a new connection every time we make a request to the server.

In my opinion, The connection should be removed from the pool once it’s closed. I am not sure if the hackney connection pool is doing this or not.

1 Like