Reliable HTTP libraries? (HTTPoison, HTTPotion, Tesla)?

There is lpgauth/buoy if you really need high performance (comes at a cost).

Or offloading the work to a more battle-tested library in another language?

I’ve only heard about puzza007/katipo which uses libcurl.

Good to know! Thanks. I’ll look into those. Might be able to accept those costs.

If you are making requests to servers running nginx (or any other sane web server), then you should be okay, I think. I’ve used buoy to interface with google cloud storage api and it worked well.

I’ve never used katipo.

The requests could be going to any server, it’s not an internal tool. Hmm.. cheers, I’ll think it over a little more. Looking at the benchmarks, there was a 0% failure rate in almost all cases, including with Hackney. Makes me think something else is up - they are using 256gb ram. One of the causes of the random timeouts in Hackney was something to do with how it was managing internal queues. So could be a memory issue.

I think I’ll have to load test these libs myself. I think that’s the only way I can be sure.

I have used HTTPoison in my production server and I am too plagued with unknown intermittent timeout issues even though I am already running the latest hackney. @BitGonzo Just wondering which library you go for in the end?

Sounds like a Hackney issue. I’ve been using Tesla and the default adaptor (httpc) and not had any problems. Have wrapped all of my requests so that I can switch out the library if it becomes a problem in the future.

Our loads will likely vary so YMMV, too.

https://github.com/teamon/tesla#adapters

I almost always start out with httpc which is built right into erlang. The api is kind of eh because it’s built for Erlang instead of Elixir, but it has never done me wrong.

It doesn’t verify certificates though last I checked?

Correct. Most HTTP clients still inherit :ssl’s default of verify: :none, meaning connections to HTTPS URLs silently ignore the server’s certificate:

iex(1)> :httpc.request('https://untrusted-root.badssl.com/')
{:ok,
 {{'HTTP/1.1', 200, 'OK'}, ...}}

At least in recent versions, when passing in verify: :verify_peer and a trust store (e.g. from the certifi package) both chain validation and hostname verification work out-of-the-box:

iex(2)> :httpc.request(:get, {'https://untrusted-root.badssl.com/', []}, [ssl: [verify: :verify_peer, cacertfile: :certifi.cacertfile]], [])
{:error,
 {:failed_connect, ...}}

iex(3)> :httpc.request(:get, {'https://wrong.host.badssl.com/', []}, [ssl: [verify: :verify_peer, cacertfile: :certifi.cacertfile]], [])    
{:error,
 {:failed_connect, ...}}

(There are some known issues with cross-signed and wildcard certs, though; I would recommend using the ssl_verify_fun package for maximum server compatibility)

Edit: the above testing was done with Erlang/OTP 21 (inets 7.0)

I am suspecting its hackney issue too, but I really like HTTPoison default ssl options and connection pooling (hackney) but the timeout issue is really a dealbreaker. httpc also seemed to be slower for me as well as my use case require me to have concurrent TCP connections.