matthias_toepp

matthias_toepp

I’m trying to get html from an old machine.

I’m able to successfully get a response on the command line with this (and I have a go program working as well):

curl 200.200.200.200/aaa/001 --user "user:password"

I get the error: “HTTP request failed: bad_request” when trying to do that with an Elixir script:

HTTPoison.start()

username = "user"
password = "password"

credentials_encoded = Base.encode64("#{username}:#{password}")

headers = [{"Authorization", "Basic #{credentials_encoded}"}]

case HTTPoison.get("200.200.200.200/aaa/001", headers) do
  {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
    IO.puts(body)
  {:ok, %HTTPoison.Response{status_code: status_code}} ->
    IO.puts("Received response with status code #{status_code}")
  {:error, %HTTPoison.Error{reason: reason}} ->
    IO.puts("HTTP request failed: #{reason}")
end

I’m able to make simple requests to other servers by altering the code slightly. I know the ip and authentication matches the working curl request, and go program.

Any advice is appreciated.

Showing Posts 1 to 8

smathy

smathy

No accept header?

al2o3cr

al2o3cr

That error comes from the Hackney machinery responsible for parsing the response; the fact that it’s going down the {:error, HTTPoison.Error} path means it’s failing to parse the opening line of the response - things like missing headers etc would generally result in an HTTPoison.Response with a status_code of 400.

The code in Hackney suggests some possiblities:

https://github.com/benoitc/hackney/blob/d6d075f9edcbb7e0b5c9418dba5dd34f2c95bc21/src/hackney_http.erl#L194-L227

  • the server is sending too many blank lines before the HTTP/1.1 200 OK
  • the server is sending malformed line-endings which cause the binary:split on line 196 to not find two parts
  • the server is sending some other kind of mis-formatted value in the opening line (omitting the reason? Spelling HTTP wrong? Hard to tell)

The most reliable way to troubleshoot this is to capture the EXACT bytes the server is replying with, for instance with curl --trace:

Matts-MacBook-Pro-2:phoenix mattjones$ curl --trace - http://example.com
== Info: Rebuilt URL to: http://example.com/
== Info:   Trying 93.184.216.34...
== Info: TCP_NODELAY set
== Info: Connected to example.com (93.184.216.34) port 80 (#0)
=> Send header, 75 bytes (0x4b)
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 48 6f 73 74 3a 20 65 78 61 6d 70 6c 65 2e 63 6f Host: example.co
0020: 6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 m..User-Agent: c
0030: 75 72 6c 2f 37 2e 35 34 2e 30 0d 0a 41 63 63 65 url/7.54.0..Acce
0040: 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a                pt: */*....
<= Recv header, 17 bytes (0x11)
0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010: 0a                                              .
<= Recv header, 13 bytes (0xd)
0000: 41 67 65 3a 20 35 30 31 35 34 32 0d 0a          Age: 501542..
<= Recv header, 31 bytes (0x1f)
0000: 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6d Cache-Control: m
0010: 61 78 2d 61 67 65 3d 36 30 34 38 30 30 0d 0a    ax-age=604800..
<= Recv header, 40 bytes (0x28)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset
0020: 3d 55 54 46 2d 38 0d 0a                         =UTF-8..
<= Recv header, 37 bytes (0x25)
0000: 44 61 74 65 3a 20 53 75 6e 2c 20 30 36 20 41 75 Date: Sun, 06 Au
0010: 67 20 32 30 32 33 20 32 30 3a 30 36 3a 35 35 20 g 2023 20:06:55 
0020: 47 4d 54 0d 0a                                  GMT..
<= Recv header, 26 bytes (0x1a)
...etcetcetc...
matthias_toepp

matthias_toepp OP

Here is the output of curl —trace (showing only the part after the send header):

== Info: Mark bundle as not supporting multiuse
== Info: HTTP 1.0, assume close after body
<= Recv header, 16 bytes (0x10)
0000: 48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f 4b 0a HTTP/1.0 200 OK.
<= Recv header, 24 bytes (0x18)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 0a                         xt/html.
<= Recv header, 1 bytes (0x1)
0000: 0a                                              .
<= Recv data, 502 bytes (0x1f6)
0000: 3c 68 74 6d 6c 3e 0a 3c 68 65 61 64 3e 0a 3c 6d <html>.<head>.<m
0010: 65 74 61 20 68 74 74 70 2d 65 71 75 69 76 3d 22 eta http-equiv="
0020: 63 6f 6e 74 65 6e 74 2d 6c 61 6e 67 75 61 67 65 content-language
0030: 22 20 63 6f 6e 74 65 6e 74 3d 22 65 6e 2d 75 73 " content="en-us
0040: 22 3e 0a 3c 6d 65 74 61 20 68 74 74 70 2d 65 71 ">.<meta http-eq
0050: 75 69 76 3d 22 63 6f 6e 74 65 6e 74 2d 74 79 70 uiv="content-typ
0060: 65 22 20 63 6f 6e 74 65 6e 74 3d 22 74 65 78 74 e" content="text
0070: 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 3d 77 /html; charset=w
0080: 69 6e 64 6f 77 73 2d 31 32 35 32 22 3e 0a 3c 6d indows-1252">.<m
0090: 65 74 61 20 68 74 74 70 2d 65 71 75 69 76 3d 22 eta http-equiv="
00a0: 70 72 61 67 6d 61 22 20 63 6f 6e 74 65 6e 74 3d pragma" content=
00b0: 22 6e 6f 2d 63 61 63 68 65 22 3e 0a 3c 74 69 74 "no-cache">.<tit
00c0: 6c 65 3e 72 61 77 20 64 61 74 61 3c 2f 74 69 74 le>raw data</tit
00d0: 6c 65 3e 0a 3c 2f 68 65 61 64 3e 0a 3c 62 6f 64 le>.</head>.<bod
00e0: 79 3e 0a 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 y>.<p><font face
00f0: 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d 34 20 63 =tahoma size=4 c
0100: 6f 6c 6f 72 3d 23 46 46 30 30 30 30 3e 3c 62 3e olor=#FF0000><b>
0110: 72 61 77 20 64 61 74 61 3c 2f 62 3e 3c 2f 66 6f raw data</b></fo
0120: 6e 74 3e 3c 2f 70 3e 0a 3c 64 69 76 20 61 6c 69 nt></p>.<div ali
0130: 67 6e 3d 22 6c 65 66 74 22 3e 0a 3c 74 61 62 6c gn="left">.<tabl
0140: 65 20 62 6f 72 64 65 72 3d 22 30 22 20 63 65 6c e border="0" cel
0150: 6c 73 70 61 63 69 6e 67 3d 22 32 22 3e 0a 3c 74 lspacing="2">.<t
0160: 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 61 63 r>.<td><font fac
0170: 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d 22 32 e=tahoma size="2
0180: 22 3e 3c 62 3e 4e 61 6d 65 26 6e 62 73 70 26 6e "><b>Name&nbsp&n
0190: 62 73 70 26 6e 62 73 70 3c 2f 62 3e 3c 2f 66 6f bsp&nbsp</b></fo
01a0: 6e 74 3e 3c 2f 74 64 3e 0a 3c 74 64 3e 3c 66 6f nt></td>.<td><fo
01b0: 6e 74 20 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 nt face=tahoma s
01c0: 69 7a 65 3d 22 32 22 3e 3c 62 3e 56 61 6c 75 65 ize="2"><b>Value
01d0: 26 6e 62 73 70 26 6e 62 73 70 26 6e 62 73 70 3c &nbsp&nbsp&nbsp<
01e0: 2f 62 3e 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a /b></font></td>.
01f0: 3c 2f 74 72 3e 0a                               </tr>.
<html>
<head>
<meta http-equiv="content-language" content="en-us">
<meta http-equiv="content-type" content="text/html; charset=windows-1252">
<meta http-equiv="pragma" content="no-cache">
<title>raw data</title>
</head>
<body>
<p><font face=tahoma size=4 color=#FF0000><b>raw data</b></font></p>
<div align="left">
<table border="0" cellspacing="2">
<tr>
<td><font face=tahoma size="2"><b>Name&nbsp&nbsp&nbsp</b></font></td>
<td><font face=tahoma size="2"><b>Value&nbsp&nbsp&nbsp</b></font></td>
</tr>
<= Recv data, 367 bytes (0x16f)
0000: 3c 74 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 <tr>.<td><font f
0010: 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d ace=tahoma size=
0020: 32 3e 6c 6f 77 5f 64 69 73 74 72 69 62 5f 70 72 2>low_distrib_pr
0030: 65 73 73 5f 61 6c 61 72 6d 26 6e 62 73 70 26 6e ess_alarm&nbsp&n
0040: 62 73 70 26 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c bsp&nbsp</font><
0050: 2f 74 64 3e 0a 3c 74 64 20 62 67 63 6f 6c 6f 72 /td>.<td bgcolor
0060: 3d 22 23 46 46 46 46 46 46 22 3e 3c 66 6f 6e 74 ="#FFFFFF"><font
0070: 20 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a  face=tahoma siz
0080: 65 3d 32 20 63 6f 6c 6f 72 3d 23 30 30 30 30 30 e=2 color=#00000
0090: 30 3e 4f 66 66 26 6e 62 73 70 26 6e 62 73 70 26 0>Off&nbsp&nbsp&
00a0: 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e nbsp</font></td>
00b0: 0a 3c 2f 74 72 3e 0a 3c 74 72 3e 0a 3c 74 64 3e .</tr>.<tr>.<td>
00c0: 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 68 6f 6d <font face=tahom
00d0: 61 20 73 69 7a 65 3d 32 3e 66 6c 74 72 5f 68 69 a size=2>fltr_hi
00e0: 67 68 5f 65 66 66 5f 74 75 72 62 5f 61 6c 61 72 gh_eff_turb_alar
00f0: 6d 26 6e 62 73 70 26 6e 62 73 70 26 6e 62 73 70 m&nbsp&nbsp&nbsp
0100: 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a 3c 74 64 </font></td>.<td
0110: 20 62 67 63 6f 6c 6f 72 3d 22 23 46 46 46 46 46  bgcolor="#FFFFF
0120: 46 22 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 F"><font face=ta
0130: 68 6f 6d 61 20 73 69 7a 65 3d 32 20 63 6f 6c 6f homa size=2 colo
0140: 72 3d 23 30 30 30 30 30 30 3e 4f 66 66 26 6e 62 r=#000000>Off&nb
0150: 73 70 26 6e 62 73 70 26 6e 62 73 70 3c 2f 66 6f sp&nbsp&nbsp</fo
0160: 6e 74 3e 3c 2f 74 64 3e 0a 3c 2f 74 72 3e 0a    nt></td>.</tr>.
<tr>
<td><font face=tahoma size=2>low_distrib_press_alarm&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<tr>
<td><font face=tahoma size=2>fltr_high_eff_turb_alarm&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<= Recv data, 182 bytes (0xb6)
0000: 3c 74 72 3e 0a 3c 74 64 3e 3c 66 6f 6e 74 20 66 <tr>.<td><font f
0010: 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 3d ace=tahoma size=
0020: 32 3e 66 6c 74 5f 68 69 67 68 5f 63 6c 65 61 72 2>flt_high_clear
0030: 77 65 6c 6c 5f 6c 76 6c 26 6e 62 73 70 26 6e 62 well_lvl&nbsp&nb
0040: 73 70 26 6e 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f sp&nbsp</font></
0050: 74 64 3e 0a 3c 74 64 20 62 67 63 6f 6c 6f 72 3d td>.<td bgcolor=
0060: 22 23 46 46 46 46 46 46 22 3e 3c 66 6f 6e 74 20 "#FFFFFF"><font
0070: 66 61 63 65 3d 74 61 68 6f 6d 61 20 73 69 7a 65 face=tahoma size
0080: 3d 32 20 63 6f 6c 6f 72 3d 23 30 30 30 30 30 30 =2 color=#000000
0090: 3e 4f 66 66 26 6e 62 73 70 26 6e 62 73 70 26 6e >Off&nbsp&nbsp&n
00a0: 62 73 70 3c 2f 66 6f 6e 74 3e 3c 2f 74 64 3e 0a bsp</font></td>.
00b0: 3c 2f 74 72 3e 0a                               </tr>.
<tr>
<td><font face=tahoma size=2>flt_high_clearwell_lvl&nbsp&nbsp&nbsp</font></td>
<td bgcolor="#FFFFFF"><font face=tahoma size=2 color=#000000>Off&nbsp&nbsp&nbsp</font></td>
</tr>
<= Recv data, 16 bytes (0x10)
0000: 3c 2f 74 61 62 6c 65 3e 0a 3c 2f 64 69 76 3e 0a </table>.</div>.
</table>
</div>
<= Recv data, 77 bytes (0x4d)
0000: 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 <p><font face=ta
0010: 68 6f 6d 61 20 73 69 7a 65 3d 32 3e 3c 62 3e 3c homa size=2><b><
0020: 61 20 68 72 65 66 3d 2e 2f 64 65 66 61 75 6c 74 a href=./default
0030: 2e 68 74 6d 3e 42 41 43 4b 3c 2f 61 3e 3c 2f 62 .htm>BACK</a></b
0040: 3e 3c 2f 66 6f 6e 74 3e 3c 2f 70 3e 0a          ></font></p>.
<p><font face=tahoma size=2><b><a href=./default.htm>BACK</a></b></font></p>
<= Recv data, 104 bytes (0x68)
0000: 3c 70 3e 3c 66 6f 6e 74 20 66 61 63 65 3d 74 61 <p><font face=ta
0010: 68 6f 6d 61 20 73 69 7a 65 3d 31 3e a9 20 31 39 homa size=1>. 19
0020: 39 33 2d 32 30 30 38 20 3c 61 20 68 72 65 66 3d 93-2008 <a href=
0030: 68 74 74 70 3a 2f 2f 77 77 77 2e 72 65 64 6c 69 http://www.redli
0040: 6f 6e 2e 6e 65 74 3e 52 65 64 20 4c 69 6f 6e 20 on.net>Red Lion
0050: 43 6f 6e 74 72 6f 6c 73 3c 2f 61 3e 3c 2f 66 6f Controls</a></fo
0060: 6e 74 3e 3c 2f 70 3e 0a                         nt></p>.
<p><font face=tahoma size=1>� 1993-2008 <a href=http://www.redlion.net>Red Lion Controls</a></font></p>
<= Recv data, 16 bytes (0x10)
0000: 3c 2f 62 6f 64 79 3e 0a 3c 2f 68 74 6d 6c 3e 0a </body>.</html>.
</body>
</html>
== Info: Closing connection 0
al2o3cr

al2o3cr

curl appears to tolerate just sending \n, but Hackney will not. The spec requires \r\n.

matthias_toepp

matthias_toepp OP

(deleted)

matthias_toepp

matthias_toepp OP

It’s kind of marvelous to me that you were able to figure that out. Thanks for the insight. I guess I’ll try a different http library then.

Maybe I’ll post this as an issue for Hackney. It kind of makes sense as apparently it’s following the spec which you linked to, but on the other hand, from my perspective, curl and go ‘worked’, in terms of getting the job done and httpoison didn’t.

matthias_toepp

matthias_toepp OP

I opened an issue relating to this on hackney’s github page:

https://github.com/benoitc/hackney/issues/723

matthias_toepp

matthias_toepp OP

Just for the record…I’m told by the maintainers that this issue is now resolved!

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New
ryanwinchester
apply_graft/2 doesn’t rewrite an add_many sub-workflow’s deps on an add step. Grafted jobs cancel with “upstream job was deleted” Version...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews