Limit number of concurrent requests to host, with Tesla

Take a look at one of the following open source libraries which can be used to rate limit requests.

• ex_rated
• hammer
• external_service

You could use the external host name as the rate limit identity. Writing your own Tesla middleware to apply the rate limit might be a good approach. Take a look at the existing fuse middleware for inspiration.

Just checking to see if there are any new opinions on this topic.

EDIT: Came across this other post, which recommends a couple resources:

• Easy and Robust Rate Limiting in Elixir-Alex Koutmos | Engineering Blog

• GitHub - uwiger/jobs: Job scheduler for load regulation · GitHub

  • A helpful comment in that other post also contains some instructions for using this library.

Follow-up: It turns out the solution I needed is built into Hackney.

To limit the maximum number of connections for a given Tesla Client (using the Hackney adapter), you can just configure Hackney’s pool and max_connections options when configuring the adapter:

  adapter({
    Tesla.Adapter.Hackney,
    pool: :some_hackney_pool,
    max_connections: 10
  })

Seems to work as expected so far. Each pool appears to work independently of the others.

To provide a bit of follow-up context from what I learned while doing this:

• This works by (ab)using the connection pool feature built into Hackney. (TLDR: HTTP connection pools save time and resources by re-using HTTP connections, which saves time doing DNS resolution, negotiating SSL/TLS security, etc.)

• Because this is not what connection pools are intended for, it is is definitely a “hack” (although I found it to be a pretty effective one).

• If you (ab)use Hackney’s connection pool settings to do this, you should probably set timeout: 0 when configuring the Hackney adapter to ensure that no connections are actually kept alive.

@arcanemachine Thanks for sharing these hacks , I am not sure I understand them though, I set new pool as:

:hackney_pool.start_pool(:my_connection_pool, [timeout: 120_000, max_connections: 100])

and then in Elixir app config I set:

config :tesla, adapter: {Tesla.Adapter.Hackney, recv_timeout: 120_000, pool: :my_connection_pool}

but when I inspect via:

iex(1)> Application.get_all_env(:hackney)
[
  timeout: 150000,
  maxr: 10,
  maxt: 1,
  restart: :permanent,
  max_connections: 50,
  shutdown: 10000
]

So it seems it does not work, how can I set my own custom pool for hackney?

Not concurrent requests in general, but for concurrent requests to the same URL I wrote a Tesla Middlewear library that handles this: