Standard HTTP Client behaviour - feature request

Hello,

I’ve created an example HTTP API library:
https://github.com/LucidModules/elixir-http
and its Hackney implementation:
https://github.com/LucidModules/elixir-http-hackney

The main advantages of this approach are:

• less dependencies - one HTTP client for all libraries, specified in the main app (in the ideal case)
• easier testing - mock response for the elixir-http library. Forget about implementation.

I will be glad to receive feedback from you.

Some thoughts in no particular order:

• in the request, headers should not be a map - order is important, and the same header can appear multiple times. Passing a map in the headers position to :hackney.request will get you a MatchError…

• in the response type, body is a map but the Hackney implementation puts the output of :hackney.body there, which is specced as binary

• loading the entire reply body into memory is probably OK, but there’s a reason that Hackney doesn’t do it automatically. Advanced use cases would want to use :hackney.stream_body or similar and there’s no way to do that with this API

• the response for connection errors is {:error, binary()} which is universal but also opaque. It’s also not satisified by the Hackney adapter, which will sometimes return errors like {:error, :connect_timeout}. Clients may want to handle situations like “no network connectivity” specifically, but that’s not possible with this interface short of pattern-matching on the error message.

• there’s no way to pass options from the caller down to the adapter, for things like proxy configuration or Hackney’s pool option. Libraries that depend on LmHttpClient should be indifferent to the adapter in use, but the end-user configuring their system is not.

• the interface forces a synchronous request, which isn’t always desirable in a BEAM program. There are alternatives nowadays like :gun or Mint that divide the work up differently.

Thank you for the fast response and your insights.

the interface forces a synchronous request

Isn’t that responsibility of the business domain to spawn new Tasks? Sometimes order of requests is important.

Neither dialyxyr nor credo found these mismatching return types you mentioned. How can we ensure the data flow integrity?

Back to the topic, do you feel that standardizing the HTTP Api for Elixir is reasonable?

I’ve created API server mock as described by Sophie DeBenedetto:
https://medium.com/flatiron-labs/rolling-your-own-mock-server-for-testing-in-elixir-2cdb5ccdd1a0

That’s good enough for me for now as the data integrity check.

After implementing a bunch of HTTP client behaviour modules for different projects and libraries I realised that vast majority of time we don’t need a module, all we need is a function. (Something to keep in mind for other things too.)

Here’s a signature:

f(options) :: {:ok, %{status:, status, headers: headers, body: body}} | {:error, exception}

where options contains :method, :url, :headers, :body and other arbitrary adapter-specific keys.

And then we’d use it like this: Foo.bar(http_request: &MyApp.http_request/1). And if we want to pass options at callsite we’d do: http_request: {&MyApp.http_request/1, receive_timeout: 1000}.

(It is pure accident that with next version of Req, an http client that I’m working on, it’d be: http_request: &Req.request/1. ;))

This is an old thread but I just saw this and it stuck out to me

Headers can be a map, but the values for duplicate fields must be combined into a comma separated list. So there should be no requirement for having headers defined as a list by either a client or server – tho the server may have to combine headers into a single value when receiving the request.