Rate limiting API with GenStage?

I have a rather expensive action (some PDF generation and upload to an other service) that occurs when related API endpoint is invoked. At the moment, there’s a pool of workers processing those requests and doing what they’re supposed to. However, this approach results in workers filling up their inboxes, potentially overwhelming the system.

The idea is to apply back pressure to API clients and just reject requests if certain conditions are met, e.g. all workers are busy.

Now, workers would be consumers within GenStage driven architecture. However, I’m at loss as to how to construct the producer part. Or, I could make it so that requests are queued somewhere and a producer that reads them, but no idea how to apply back pressure to API clients.

https://github.com/loucash/eqm solves this idea in the sense that I can use it in e.g. endpoint controller to dispatch work to workers and get information if the pool is exhausted or not, consequently informing the API client. So I’m looking for a similar solution via GenStage, if applicable.

Thank you for your time!

1 Like

Look at the buffering demand part of the GenStage docs https://hexdocs.pm/gen_stage/GenStage.html. It includes a QueueBroadcaster which shows how to manage demand and back pressure. In the case of a web API the caller can make calls any time they choose. You would need to provide some reply they interpret as rejecting input that they would be responsible for resubmitting at a later time. This would be similar to how the public APIs like google provide rate limits. The request gets an error that is interpreted as hitting a limit.

1 Like

There’s also an example in the gen_stage repo for rate limiting that you might want to check out.

1 Like

I gave it a look before, it does show how to rate limit from consumer perspective, but didn’t help with the problem of informing the clients of that rate limit. Clients in this case come even before producer. Maybe I missed something, will give it another look. Thanks!

Excellent, seems to be on the right track.
Thank you very much!