Empty response

Questions / Help
, ,
1

I’d like to scrape a website but for some reason, HTTPoison gives me an empty body.

iex> HTTPoison.get(“http://www.origo.hu/contentpartner/rss/origoall/origo.xml”)
{:ok,
%HTTPoison.Response{
body: “”,
headers: [
{“Content-length”, “0”},
{“Location”, “https://www.origo.hu/contentpartner/rss/origoall/origo.xml”}
],
request: %HTTPoison.Request{
body: “”,
headers: [],
method: :get,
options: [],
params: %{},
url: “http://www.origo.hu/contentpartner/rss/origoall/origo.xml”
},
request_url: “http://www.origo.hu/contentpartner/rss/origoall/origo.xml”,
status_code: 301
}}

I tried it with HTTPotion too, the same result:
%HTTPotion.Response{
body: “”,
headers: %HTTPotion.Headers{
hdrs: %{
“content-length” => “0”,
“location” => “https://www.origo.hu/contentpartner/rss/origoall/origo.xml”
}
},
status_code: 301
}

I checked the site, it is definitely not empty. I tried to fetch it with Nodejs and it works with that.

As far as I understand both of these clients use Hackney and there is an issue about this on the HTTPoisonGithub. But this issue is closed, so I guess this problem shouldn’t exist anymore?
Does anyone have any idea what should I do here?

2

So you are getting a 301 redirect, precisely you are being redirected from http:// to https://.

You can use just http:// in the original URL, or configure HTTPoison to follow redirects like this: httpoison/test/httpoison_test.exs at 3481d567adcb06d129dfec3c69aa87d1719f7a8b · edgurgel/httpoison · GitHub

4 Likes
3

OMG. Feel so stupid right now. Thank you so much! I don’t know how I missed the status code…

1 Like
4

That happens a lot, sometimes you just end up writing question or explain problem to someone and solution comes in, sometimes you actually need 2nd pair of eyes to spot obvious.

2 Likes