In the library I was writing I had to implement a rate limiter to limit the amount of calls going out to an external API.
The design I eventually came up with is following:

• Ratelimit process has an ETS with the actual state of the rate limit per bucket (remaining, wait time, …)
• Poolboy manages a set of processes that can execute the HTTP request based on the state (they receive the state along the HTTP call they need to make)

The Ratelimit process receives ALL calls, looks up the state for a bucket and sends it the API call to execute along with the state.

The problem was that API calls often have a result you want to return, so you’d use a call instead of a cast. But you don’t want the Ratelimit process to be blocked until the API call finishes (especially if it’s going to have to wait with executing).

As a solution I “forward” the call to the bucket which handles the actual response to the original caller with the following code:

  # This method forwards a call to another genserver.
  # We use this method to forward a request to the ratelimit to the bucket which will then handle it.
  # Forwarding instead of simply calling GenServer.call to the bucket allows the ratelimit to continue processing without blocking.
  @spec forward_call(server :: GenServer.server(), event :: any(), from :: GenServer.from()) ::
          :ok
  defp forward_call(server, event, from) do
    target = GenServer.whereis(server)

    send(target, {:"$gen_call", from, event})
    :ok
  end

For the caller this is entirely transparent, and the bucket can just {:reply} as it always has, but I feel like this is a little bit of a hack (and I haven’t seen this technique used anywhere else in the ecosystem).

What do you guys think of this approach, did I just go about the problem entirely wrong or is this a clever solution that you would’ve used as well?

EDIT:
link to the code: wumpex/lib/wumpex/api/ratelimit.ex at master · dealloc/wumpex · GitHub
bucket lookup: wumpex/lib/wumpex/api/ratelimit.ex at master · dealloc/wumpex · GitHub
bucket code: wumpex/lib/wumpex/api/ratelimit/stateless_bucket.ex at master · dealloc/wumpex · GitHub