matthias_toepp

Httpoison get request with basic authorization -- "HTTP request failed: bad_request"

I’m trying to get html from an old machine.

I’m able to successfully get a response on the command line with this (and I have a go program working as well):

curl 200.200.200.200/aaa/001 --user "user:password"

I get the error: “HTTP request failed: bad_request” when trying to do that with an Elixir script:

HTTPoison.start()

username = "user"
password = "password"

credentials_encoded = Base.encode64("#{username}:#{password}")

headers = [{"Authorization", "Basic #{credentials_encoded}"}]

case HTTPoison.get("200.200.200.200/aaa/001", headers) do
  {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
    IO.puts(body)
  {:ok, %HTTPoison.Response{status_code: status_code}} ->
    IO.puts("Received response with status code #{status_code}")
  {:error, %HTTPoison.Error{reason: reason}} ->
    IO.puts("HTTP request failed: #{reason}")
end

I’m able to make simple requests to other servers by altering the code slightly. I know the ip and authentication matches the working curl request, and go program.

Any advice is appreciated.

8 comments

#http

0 1469 8

2023-10-19 13:21:53 UTC

Marked As Solved

al2o3cr

curl appears to tolerate just sending \n, but Hackney will not. The spec requires \r\n.

Post #5

Also Liked

al2o3cr

That error comes from the Hackney machinery responsible for parsing the response; the fact that it’s going down the {:error, HTTPoison.Error} path means it’s failing to parse the opening line of the response - things like missing headers etc would generally result in an HTTPoison.Response with a status_code of 400.

The code in Hackney suggests some possiblities:

https://github.com/benoitc/hackney/blob/d6d075f9edcbb7e0b5c9418dba5dd34f2c95bc21/src/hackney_http.erl#L194-L227

the server is sending too many blank lines before the HTTP/1.1 200 OK
the server is sending malformed line-endings which cause the binary:split on line 196 to not find two parts
the server is sending some other kind of mis-formatted value in the opening line (omitting the reason? Spelling HTTP wrong? Hard to tell)

The most reliable way to troubleshoot this is to capture the EXACT bytes the server is replying with, for instance with curl --trace:

Matts-MacBook-Pro-2:phoenix mattjones$ curl --trace - http://example.com
== Info: Rebuilt URL to: http://example.com/
== Info:   Trying 93.184.216.34...
== Info: TCP_NODELAY set
== Info: Connected to example.com (93.184.216.34) port 80 (#0)
=> Send header, 75 bytes (0x4b)
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 48 6f 73 74 3a 20 65 78 61 6d 70 6c 65 2e 63 6f Host: example.co
0020: 6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 m..User-Agent: c
0030: 75 72 6c 2f 37 2e 35 34 2e 30 0d 0a 41 63 63 65 url/7.54.0..Acce
0040: 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a                pt: */*....
<= Recv header, 17 bytes (0x11)
0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010: 0a                                              .
<= Recv header, 13 bytes (0xd)
0000: 41 67 65 3a 20 35 30 31 35 34 32 0d 0a          Age: 501542..
<= Recv header, 31 bytes (0x1f)
0000: 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6d Cache-Control: m
0010: 61 78 2d 61 67 65 3d 36 30 34 38 30 30 0d 0a    ax-age=604800..
<= Recv header, 40 bytes (0x28)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset
0020: 3d 55 54 46 2d 38 0d 0a                         =UTF-8..
<= Recv header, 37 bytes (0x25)
0000: 44 61 74 65 3a 20 53 75 6e 2c 20 30 36 20 41 75 Date: Sun, 06 Au
0010: 67 20 32 30 32 33 20 32 30 3a 30 36 3a 35 35 20 g 2023 20:06:55 
0020: 47 4d 54 0d 0a                                  GMT..
<= Recv header, 26 bytes (0x1a)
...etcetcetc...

Post #3