Standard HTTP Client behaviour - feature request

Hello,

I am working on a SSO implementation for Line Login and Ueberauth. New version was released but the current maintainer did not update API for Line Login v2.1.

What I’ve found is that each Ueberauth SSO implementation can use it’s very own HTTP client.

I feel is that developer should be able to inject desired HTTP client for all Elixir modules.
Unfortunately, there is no such configuration yet.

This forum has a topic about standard HTTP library implemented in the Elixir and disadvantages of multiple clients for each library:

However, as michalmuskala pointed it would require maintenance by the Elixir team.

I’m coming from the OOP world and dependency inversion is common here. For example, PHP has the PSR - PHP standard recommendations - interfaces defining a contract between libraries and applications.
Here is the example: PSR-18: HTTP Client - PHP-FIG
Another example for Logger: PSR-3: Logger Interface - PHP-FIG

Is it possible to have one standarized client API, e.g. a behaviour, that could be implemented by various HTTP clients? I am curious if this methodology is in line with the Elixir/functional philosophy.

1 Like

Hello,

I’ve created an example HTTP API library:

and its Hackney implementation:

The main advantages of this approach are:

  • less dependencies - one HTTP client for all libraries, specified in the main app (in the ideal case)
  • easier testing - mock response for the elixir-http library. Forget about implementation.

I will be glad to receive feedback from you.

Some thoughts in no particular order:

  • in the request, headers should not be a map - order is important, and the same header can appear multiple times. Passing a map in the headers position to :hackney.request will get you a MatchError

  • in the response type, body is a map but the Hackney implementation puts the output of :hackney.body there, which is specced as binary

  • loading the entire reply body into memory is probably OK, but there’s a reason that Hackney doesn’t do it automatically. Advanced use cases would want to use :hackney.stream_body or similar and there’s no way to do that with this API

  • the response for connection errors is {:error, binary()} which is universal but also opaque. It’s also not satisified by the Hackney adapter, which will sometimes return errors like {:error, :connect_timeout}. Clients may want to handle situations like “no network connectivity” specifically, but that’s not possible with this interface short of pattern-matching on the error message.

  • there’s no way to pass options from the caller down to the adapter, for things like proxy configuration or Hackney’s pool option. Libraries that depend on LmHttpClient should be indifferent to the adapter in use, but the end-user configuring their system is not.

  • the interface forces a synchronous request, which isn’t always desirable in a BEAM program. There are alternatives nowadays like :gun or Mint that divide the work up differently.

6 Likes

Thank you for the fast response and your insights.

the interface forces a synchronous request

Isn’t that responsibility of the business domain to spawn new Tasks? Sometimes order of requests is important.

Neither dialyxyr nor credo found these mismatching return types you mentioned. How can we ensure the data flow integrity?

Back to the topic, do you feel that standardizing the HTTP Api for Elixir is reasonable?

1 Like

I’ve created API server mock as described by Sophie DeBenedetto:

That’s good enough for me for now as the data integrity check.

1 Like

After implementing a bunch of HTTP client behaviour modules for different projects and libraries I realised that vast majority of time we don’t need a module, all we need is a function. (Something to keep in mind for other things too.)

Here’s a signature:

f(options) :: {:ok, %{status:, status, headers: headers, body: body}} | {:error, exception}

where options contains :method, :url, :headers, :body and other arbitrary adapter-specific keys.

And then we’d use it like this: Foo.bar(http_request: &MyApp.http_request/1). And if we want to pass options at callsite we’d do: http_request: {&MyApp.http_request/1, receive_timeout: 1000}.

(It is pure accident that with next version of Req, an http client that I’m working on, it’d be: http_request: &Req.request/1. ;))

9 Likes

This is an old thread but I just saw this and it stuck out to me :slight_smile:

Headers can be a map, but the values for duplicate fields must be combined into a comma separated list. So there should be no requirement for having headers defined as a list by either a client or server – tho the server may have to combine headers into a single value when receiving the request.

1 Like