matthias_toepp

matthias_toepp

I’m trying to get html from an old machine.

I’m able to successfully get a response on the command line with this (and I have a go program working as well):

curl 200.200.200.200/aaa/001 --user "user:password"

I get the error: “HTTP request failed: bad_request” when trying to do that with an Elixir script:

HTTPoison.start()

username = "user"
password = "password"

credentials_encoded = Base.encode64("#{username}:#{password}")

headers = [{"Authorization", "Basic #{credentials_encoded}"}]

case HTTPoison.get("200.200.200.200/aaa/001", headers) do
  {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
    IO.puts(body)
  {:ok, %HTTPoison.Response{status_code: status_code}} ->
    IO.puts("Received response with status code #{status_code}")
  {:error, %HTTPoison.Error{reason: reason}} ->
    IO.puts("HTTP request failed: #{reason}")
end

I’m able to make simple requests to other servers by altering the code slightly. I know the ip and authentication matches the working curl request, and go program.

Any advice is appreciated.

Showing Posts 1 to 8

smathy

smathy

No accept header?

al2o3cr

al2o3cr

That error comes from the Hackney machinery responsible for parsing the response; the fact that it’s going down the {:error, HTTPoison.Error} path means it’s failing to parse the opening line of the response - things like missing headers etc would generally result in an HTTPoison.Response with a status_code of 400.

The code in Hackney suggests some possiblities:

https://github.com/benoitc/hackney/blob/d6d075f9edcbb7e0b5c9418dba5dd34f2c95bc21/src/hackney_http.erl#L194-L227

  • the server is sending too many blank lines before the HTTP/1.1 200 OK
  • the server is sending malformed line-endings which cause the binary:split on line 196 to not find two parts
  • the server is sending some other kind of mis-formatted value in the opening line (omitting the reason? Spelling HTTP wrong? Hard to tell)

The most reliable way to troubleshoot this is to capture the EXACT bytes the server is replying with, for instance with curl --trace:

Matts-MacBook-Pro-2:phoenix mattjones$ curl --trace - http://example.com
== Info: Rebuilt URL to: http://example.com/
== Info:   Trying 93.184.216.34...
== Info: TCP_NODELAY set
== Info: Connected to example.com (93.184.216.34) port 80 (#0)
=> Send header, 75 bytes (0x4b)
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 48 6f 73 74 3a 20 65 78 61 6d 70 6c 65 2e 63 6f Host: example.co
0020: 6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 m..User-Agent: c
0030: 75 72 6c 2f 37 2e 35 34 2e 30 0d 0a 41 63 63 65 url/7.54.0..Acce
0040: 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a                pt: */*....
<= Recv header, 17 bytes (0x11)
0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010: 0a                                              .
<= Recv header, 13 bytes (0xd)
0000: 41 67 65 3a 20 35 30 31 35 34 32 0d 0a          Age: 501542..
<= Recv header, 31 bytes (0x1f)
0000: 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6d Cache-Control: m
0010: 61 78 2d 61 67 65 3d 36 30 34 38 30 30 0d 0a    ax-age=604800..
<= Recv header, 40 bytes (0x28)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset
0020: 3d 55 54 46 2d 38 0d 0a                         =UTF-8..
<= Recv header, 37 bytes (0x25)
0000: 44 61 74 65 3a 20 53 75 6e 2c 20 30 36 20 41 75 Date: Sun, 06 Au
0010: 67 20 32 30 32 33 20 32 30 3a 30 36 3a 35 35 20 g 2023 20:06:55 
0020: 47 4d 54 0d 0a                                  GMT..
<= Recv header, 26 bytes (0x1a)
...etcetcetc...
matthias_toepp

matthias_toepp OP

Here is the output of curl —trace (showing only the part after the send header):

== Info: Mark bundle as not supporting multiuse
== Info: HTTP 1.0, assume close after body
<= Recv header, 16 bytes (0x10)
0000: 48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f 4b 0a HTTP/1.0 200 OK.
<= Recv header, 24 bytes (0x18)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 0a                         xt/html.
<= Recv header, 1 bytes (0x1)
0000: 0a                                              .
<= Recv data, 502 bytes (0x1f6)
0000: 3c 68 74 6d 6c 3e 0a 3c 68 65 61 64 3e 0a 3c 6d <html>.<head>.<m
0010: 65 74 61 20 68 74 74 70 2d 65 71 75 69 76 3d 22 eta http-equiv="
0020: 63 6f 6e 74 65 6e 74 2d 6c 61 6e 67 75 61 67 65 content-language
0030: 22 20 63 6f 6e 74 65 6e 74 3d 22 65 6e 2d 75 73 " content="en-us
0040: 22 3e 0a 3c 6d 65 74 61 20 68 74 74 70 2d 65 71 ">.<meta http-eq
0050: 75 69 76 3d 22 63 6f 6e 74 65 6e 74 2d 74 79 70 uiv="content-typ
0060: 65 22 20 63 6f 6e 74 65 6e 74 3d 22 74 65 78 74 e" content="text
0070: 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 3d 77 /html; charset=w
0080: 69 6e 64 6f 77 73 2d 31 32 35 32 22 3e 0a 3c 6d indows-1252">.<m
0090: 65 74 61 20 68 74 74 70 2d 65 71 75 69 76 3d 22 eta http-equiv="
00a0: 70 72 61 67 6d 61 22 20 63 6f 6e 74 65 6e 74 3d pragma" content=
00b0: 22 6e 6f 2d 63 61 63 68 65 22 3e 0a 3c 74 69 74 "no-cache">.<tit
00c0: 6c 65 3e 72 61 77 20 64 61 74 61 3c 2f 74 69 74 le>raw data</tit
00d0: 6c 65 3e 0a 3c 2f 68 65 61 64 3e 0a 3c 62 6f 64 le>.</head>.<bod
00e0: 79 3e 0a 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 y>.<p><font face
00f0: 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d 34 20 63 =tahoma size=4 c
0100: 6f 6c 6f 72 3d 23 46 46 30 30 30 30 3e 3c 62 3e olor=#FF0000><b>
0110: 72 61 77 20 64 61 74 61 3c 2f 62 3e 3c 2f 66 6f raw data</b></fo
0120: 6e 74 3e 3c 2f 70 3e 0a 3c 64 69 76 20 61 6c 69 nt></p>.<div ali
0130: 67 6e 3d 22 6c 65 66 74 22 3e 0a 3c 74 61 62 6c gn="left">.<tabl
0140: 65 20 62 6f 72 64 65 72 3d 22 30 22 20 63 65 6c e border="0" cel
0150: 6c 73 70 61 63 69 6e 67 3d 22 32 22 3e 0a 3c 74 lspacing="2">.<t
0160: 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 61 63 r>.<td><font fac
0170: 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d 22 32 e=tahoma size="2
0180: 22 3e 3c 62 3e 4e 61 6d 65 26 6e 62 73 70 26 6e "><b>Name&nbsp&n
0190: 62 73 70 26 6e 62 73 70 3c 2f 62 3e 3c 2f 66 6f bsp&nbsp</b></fo
01a0: 6e 74 3e 3c 2f 74 64 3e 0a 3c 74 64 3e 3c 66 6f nt></td>.<td><fo
01b0: 6e 74 20 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 nt face=tahoma s
01c0: 69 7a 65 3d 22 32 22 3e 3c 62 3e 56 61 6c 75 65 ize="2"><b>Value
01d0: 26 6e 62 73 70 26 6e 62 73 70 26 6e 62 73 70 3c &nbsp&nbsp&nbsp<
01e0: 2f 62 3e 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a /b></font></td>.
01f0: 3c 2f 74 72 3e 0a                               </tr>.
<html>
<head>
<meta http-equiv="content-language" content="en-us">
<meta http-equiv="content-type" content="text/html; charset=windows-1252">
<meta http-equiv="pragma" content="no-cache">
<title>raw data</title>
</head>
<body>
<p><font face=tahoma size=4 color=#FF0000><b>raw data</b></font></p>
<div align="left">
<table border="0" cellspacing="2">
<tr>
<td><font face=tahoma size="2"><b>Name&nbsp&nbsp&nbsp</b></font></td>
<td><font face=tahoma size="2"><b>Value&nbsp&nbsp&nbsp</b></font></td>
</tr>
<= Recv data, 367 bytes (0x16f)
0000: 3c 74 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 <tr>.<td><font f
0010: 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d ace=tahoma size=
0020: 32 3e 6c 6f 77 5f 64 69 73 74 72 69 62 5f 70 72 2>low_distrib_pr
0030: 65 73 73 5f 61 6c 61 72 6d 26 6e 62 73 70 26 6e ess_alarm&nbsp&n
0040: 62 73 70 26 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c bsp&nbsp</font><
0050: 2f 74 64 3e 0a 3c 74 64 20 62 67 63 6f 6c 6f 72 /td>.<td bgcolor
0060: 3d 22 23 46 46 46 46 46 46 22 3e 3c 66 6f 6e 74 ="#FFFFFF"><font
0070: 20 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a  face=tahoma siz
0080: 65 3d 32 20 63 6f 6c 6f 72 3d 23 30 30 30 30 30 e=2 color=#00000
0090: 30 3e 4f 66 66 26 6e 62 73 70 26 6e 62 73 70 26 0>Off&nbsp&nbsp&
00a0: 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e nbsp</font></td>
00b0: 0a 3c 2f 74 72 3e 0a 3c 74 72 3e 0a 3c 74 64 3e .</tr>.<tr>.<td>
00c0: 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 68 6f 6d <font face=tahom
00d0: 61 20 73 69 7a 65 3d 32 3e 66 6c 74 72 5f 68 69 a size=2>fltr_hi
00e0: 67 68 5f 65 66 66 5f 74 75 72 62 5f 61 6c 61 72 gh_eff_turb_alar
00f0: 6d 26 6e 62 73 70 26 6e 62 73 70 26 6e 62 73 70 m&nbsp&nbsp&nbsp
0100: 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a 3c 74 64 </font></td>.<td
0110: 20 62 67 63 6f 6c 6f 72 3d 22 23 46 46 46 46 46  bgcolor="#FFFFF
0120: 46 22 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 F"><font face=ta
0130: 68 6f 6d 61 20 73 69 7a 65 3d 32 20 63 6f 6c 6f homa size=2 colo
0140: 72 3d 23 30 30 30 30 30 30 3e 4f 66 66 26 6e 62 r=#000000>Off&nb
0150: 73 70 26 6e 62 73 70 26 6e 62 73 70 3c 2f 66 6f sp&nbsp&nbsp</fo
0160: 6e 74 3e 3c 2f 74 64 3e 0a 3c 2f 74 72 3e 0a    nt></td>.</tr>.
<tr>
<td><font face=tahoma size=2>low_distrib_press_alarm&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<tr>
<td><font face=tahoma size=2>fltr_high_eff_turb_alarm&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<= Recv data, 182 bytes (0xb6)
0000: 3c 74 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 <tr>.<td><font f
0010: 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d ace=tahoma size=
0020: 32 3e 66 6c 74 5f 68 69 67 68 5f 63 6c 65 61 72 2>flt_high_clear
0030: 77 65 6c 6c 5f 6c 76 6c 26 6e 62 73 70 26 6e 62 well_lvl&nbsp&nb
0040: 73 70 26 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f sp&nbsp</font></
0050: 74 64 3e 0a 3c 74 64 20 62 67 63 6f 6c 6f 72 3d td>.<td bgcolor=
0060: 22 23 46 46 46 46 46 46 22 3e 3c 66 6f 6e 74 20 "#FFFFFF"><font
0070: 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 face=tahoma size
0080: 3d 32 20 63 6f 6c 6f 72 3d 23 30 30 30 30 30 30 =2 color=#000000
0090: 3e 4f 66 66 26 6e 62 73 70 26 6e 62 73 70 26 6e >Off&nbsp&nbsp&n
00a0: 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a bsp</font></td>.
00b0: 3c 2f 74 72 3e 0a                               </tr>.
<tr>
<td><font face=tahoma size=2>flt_high_clearwell_lvl&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<= Recv data, 16 bytes (0x10)
0000: 3c 2f 74 61 62 6c 65 3e 0a 3c 2f 64 69 76 3e 0a </table>.</div>.
</table>
</div>
<= Recv data, 77 bytes (0x4d)
0000: 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 <p><font face=ta
0010: 68 6f 6d 61 20 73 69 7a 65 3d 32 3e 3c 62 3e 3c homa size=2><b><
0020: 61 20 68 72 65 66 3d 2e 2f 64 65 66 61 75 6c 74 a href=./default
0030: 2e 68 74 6d 3e 42 41 43 4b 3c 2f 61 3e 3c 2f 62 .htm>BACK</a></b
0040: 3e 3c 2f 66 6f 6e 74 3e 3c 2f 70 3e 0a          ></font></p>.
<p><font face=tahoma size=2><b><a href=./default.htm>BACK</a></b></font></p>
<= Recv data, 104 bytes (0x68)
0000: 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 <p><font face=ta
0010: 68 6f 6d 61 20 73 69 7a 65 3d 31 3e a9 20 31 39 homa size=1>. 19
0020: 39 33 2d 32 30 30 38 20 3c 61 20 68 72 65 66 3d 93-2008 <a href=
0030: 68 74 74 70 3a 2f 2f 77 77 77 2e 72 65 64 6c 69 http://www.redli
0040: 6f 6e 2e 6e 65 74 3e 52 65 64 20 4c 69 6f 6e 20 on.net>Red Lion
0050: 43 6f 6e 74 72 6f 6c 73 3c 2f 61 3e 3c 2f 66 6f Controls</a></fo
0060: 6e 74 3e 3c 2f 70 3e 0a                         nt></p>.
<p><font face=tahoma size=1>� 1993-2008 <a href=http://www.redlion.net>Red Lion Controls</a></font></p>
<= Recv data, 16 bytes (0x10)
0000: 3c 2f 62 6f 64 79 3e 0a 3c 2f 68 74 6d 6c 3e 0a </body>.</html>.
</body>
</html>
== Info: Closing connection 0
al2o3cr

al2o3cr

curl appears to tolerate just sending \n, but Hackney will not. The spec requires \r\n.

matthias_toepp

matthias_toepp OP

(deleted)

matthias_toepp

matthias_toepp OP

It’s kind of marvelous to me that you were able to figure that out. Thanks for the insight. I guess I’ll try a different http library then.

Maybe I’ll post this as an issue for Hackney. It kind of makes sense as apparently it’s following the spec which you linked to, but on the other hand, from my perspective, curl and go ‘worked’, in terms of getting the job done and httpoison didn’t.

matthias_toepp

matthias_toepp OP

I opened an issue relating to this on hackney’s github page:

https://github.com/benoitc/hackney/issues/723

matthias_toepp

matthias_toepp OP

Just for the record…I’m told by the maintainers that this issue is now resolved!

— All posts loaded —

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
itzmidinesh
Hi all, I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
yarrichar
I’m very new to Elixir, so sorry if this is a silly question: I am wondering what happens to the mailbox of a process when the machine i...
New
pragdave
How do I tell Hologram not to escape HTML when substituting into templates? {Textile.render(@product.long_description)} escapes the HTM...
New
Fl4m3Ph03n1x
Background When trying to execute mix release on a Windows 11 machine for a Phoenix project I get the following error: * assembling mark...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New
rlipscombe
I’d like to create a file with 600 permissions (rw-------), but there’s a race condition if I create it and then change the permissions. ...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews