Limit number of concurrent requests to host, with Tesla

Questions / Help
,
1

Hello, I’m using the tesla library to send http requests to a host. However I’m supposed to keep the amount of concurrent requests somewhat low.

To achieve this I used poolboy. The problem with this approach however is that I have to use mock_global during testing, which is not a problem for now, but could become one later on.

Is there a better alternative to this approach? My current telsa adapter is hackney, but it doesn’t matter, if I have to switch.

2

Take a look at one of the following open source libraries which can be used to rate limit requests.

You could use the external host name as the rate limit identity. Writing your own Tesla middleware to apply the rate limit might be a good approach. Take a look at the existing fuse middleware for inspiration.

6 Likes
3

Just checking to see if there are any new opinions on this topic. :wave:

EDIT: Came across this other post, which recommends a couple resources:

4

Follow-up: It turns out the solution I needed is built into Hackney.

To limit the maximum number of connections for a given Tesla Client (using the Hackney adapter), you can just configure Hackney’s pool and max_connections options when configuring the adapter:

  adapter({
    Tesla.Adapter.Hackney,
    pool: :some_hackney_pool,
    max_connections: 10
  })

Seems to work as expected so far. Each pool appears to work independently of the others. :slight_smile:

5

To provide a bit of follow-up context from what I learned while doing this:

  • This works by (ab)using the connection pool feature built into Hackney. (TLDR: HTTP connection pools save time and resources by re-using HTTP connections, which saves time doing DNS resolution, negotiating SSL/TLS security, etc.)

  • Because this is not what connection pools are intended for, it is is definitely a “hack” (although I found it to be a pretty effective one).

  • If you (ab)use Hackney’s connection pool settings to do this, you should probably set timeout: 0 when configuring the Hackney adapter to ensure that no connections are actually kept alive.

6

@arcanemachine Thanks for sharing these hacks :slight_smile:, I am not sure I understand them though, I set new pool as:

:hackney_pool.start_pool(:my_connection_pool, [timeout: 120_000, max_connections: 100])

and then in Elixir app config I set:

config :tesla, adapter: {Tesla.Adapter.Hackney, recv_timeout: 120_000, pool: :my_connection_pool}

but when I inspect via:

iex(1)> Application.get_all_env(:hackney)
[
  timeout: 150000,
  maxr: 10,
  maxt: 1,
  restart: :permanent,
  max_connections: 50,
  shutdown: 10000
]

So it seems it does not work, how can I set my own custom pool for hackney?

7

Not concurrent requests in general, but for concurrent requests to the same URL I wrote a Tesla Middlewear library that handles this:

2 Likes