Rather than conflating rate limiter with the pool, you could have a simple rate limiter which responds with yes/no. Then, each client process (in your case resource worker) would ask the limiter for permission. If the permission is given, the client can safely issue the API request (or any other limited job).

This means you don’t need a pool at all. Instead, the rate limiter ensures that at most n API requests are issued in a given time frame. You won’t be able to do more, since limiter won’t allow it.

There are a couple of libraries for this available. Quick googling revealed this one. I never tried it so can’t comment how it works. For my blog, I implemented a naive solution based on ETS counters. This one’s nice because permission questions are not going through the single process, so it should have better throughput. However, since it’ based on ETS tables, it can’t work as a cluster-wide (global) limiter.

It’s worth mentioning that these rate limiters are not 100% consistent, because things happen concurrently. Namely, when a limiter gives you permission, you still need to make the request. So I think it’s theoretically possible that in one second you end up issuing a bit more requests than allowed. A simple remedy for this would be to set the internal limit a bit lower than the one imposed by the external API. If you really want strong consistency I think the only option is to forward all calls through the single process. This would provide stronger guarantees, but might affect the throughput.

GitHub - jlouis/safetyvalve: A safety valve for your erlang node · GitHub provides exactly these features.

However sbroker master also provides a framework for building custom job regulators, there is even a property testing framework for the testing it. Its Erlang code but the simplest one limits the number of concurrent tasks: sbroker/src/sregulator_open_valve.erl at master · fishcakez/sbroker · GitHub. The regulator process plugs together a valve, a queue and a meter, and has 3 different ways to request to run, blocking with queue, blocking without queueing and asynchronous with queue. The queues and meters come with sbroker but you would need to write the valve. I would really like a PR adding this feature and can help with the property based testing.

If you use GenServer.call to issue a request to the rate limiter it will simplify some things. In particular, you won’t need to monitor R from C (GenServer.call does that for you). You also don’t need to monitor C from R if you can immediately respond.

To sketch the idea, handle_call could look something like:

def handle_call(:request_permission, from, state) do
  if can_grant?(state) do
    {:reply, :ok, inc_issued_count(state)}
  else
    {:noreply, enqueue_caller(state, from)}
  end
end

Where enqueue_caller would have to store the from tuple and setup a monitor to the caller, whose pid is the first element of the from tuple (see here).

Then, when you want to respond to the caller at the later point, you can use GenServer.reply, passing the dequeued from tuple. When doing this, you should also demonitor the corresponding caller.

I think you could also make it work without using Process.send_after, by relying on timeout values in response tuples and some juggling with monotonic time. This would allow you to reset the counter even when you’re highly loaded, because you could also reset it when handling an incoming request. However, that would probably be a more complex solution, so I’d start with send_after, and then maybe refine once everything else is in place.

I think that’s the general idea