dealloc
In the library I was writing I had to implement a rate limiter to limit the amount of calls going out to an external API.
The design I eventually came up with is following:
- Ratelimit process has an ETS with the actual state of the rate limit per bucket (remaining, wait time, …)
- Poolboy manages a set of processes that can execute the HTTP request based on the state (they receive the state along the HTTP call they need to make)
The Ratelimit process receives ALL calls, looks up the state for a bucket and sends it the API call to execute along with the state.
The problem was that API calls often have a result you want to return, so you’d use a call instead of a cast. But you don’t want the Ratelimit process to be blocked until the API call finishes (especially if it’s going to have to wait with executing).
As a solution I “forward” the call to the bucket which handles the actual response to the original caller with the following code:
# This method forwards a call to another genserver.
# We use this method to forward a request to the ratelimit to the bucket which will then handle it.
# Forwarding instead of simply calling GenServer.call to the bucket allows the ratelimit to continue processing without blocking.
@spec forward_call(server :: GenServer.server(), event :: any(), from :: GenServer.from()) ::
:ok
defp forward_call(server, event, from) do
target = GenServer.whereis(server)
send(target, {:"$gen_call", from, event})
:ok
end
For the caller this is entirely transparent, and the bucket can just {:reply} as it always has, but I feel like this is a little bit of a hack (and I haven’t seen this technique used anywhere else in the ecosystem).
What do you guys think of this approach, did I just go about the problem entirely wrong or is this a clever solution that you would’ve used as well?
EDIT:
link to the code: wumpex/lib/wumpex/api/ratelimit.ex at master · dealloc/wumpex · GitHub
bucket lookup: wumpex/lib/wumpex/api/ratelimit.ex at master · dealloc/wumpex · GitHub
bucket code: wumpex/lib/wumpex/api/ratelimit/stateless_bucket.ex at master · dealloc/wumpex · GitHub
Where Next?
Trending in Discussions
Other Trending Topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 6 of 6 Posts
Matsa59
Hello,
GenServer is really fun to use but in this case you should avoid them. It will be the bottleneck of your whole system quickly.
You should consider using ets only and you don’t need to wrap it under a specific process. Take a look at ets update counter (ets — OTP 29.0.2 (stdlib 8.0.1))
(Sorry it’s Erlang doc, but never too late/soon to learn Erlang)
For the rest of the code stay in the http process.
I hope it help you
dealloc
Given that the StatelessBucket processes don’t keep state themselves I can see why you would suggest that. However, I chose using GenServer since that still allows using the
handle_call/3callbacks (so I can reply to the call method without having to simulate a reply myself).I don’t see how GenServer would be a bottleneck here though?
The ETS update_counter is pretty good, I’ll definitely look into that. It seems perfectly suited for what I’m doing with it here.
Matsa59
A simple representation of a GenServer is a loop that dequeue messages.
Imagine that you receive a lot of requests, so many that your GenServer could not dequeue them as fast as he need. The messages box will growing and could reach a point where new messages will be process 2 minutes later.
In this case, what should we do for those requests ? In general case, they will timeout. That’s why I prefer use ets in your case
Other concern: Sometime we don’t have choice to use GenServer, but do they have to be unique? There is a lot of tips to manage multiple Genservers at the same time.
Edit: currently you don’t use correctly the GenServer (IMO). They were created to manage state. Your is stateless and is use to manage a async job that will return something.
dealloc
The growing messagebox will hold true for any type of Elixir process though?
What would you suggest then, not running the API call themselves in a separate process?
The state is currently already tracked in ETS which is shared over a pool of GenServer, so they aren’t even unique to start with.
The only exception being the Ratelimit, which does nothing more than an ETS lookup and then “forwards” the call to the bucket (and that’s what the question was about)
Matsa59
Yeah I talk about the RateLimit, you could simply have a module that check ets and forward to what ever you want.
Thing is if you don’t use GenServer for this module, it will be executed directly in the client request process. So you’ll have more processes that will « do the job ».
Lot of people wrap ets inside a GenServer and it an error IMO
dealloc
Ah, now I’m following.
I’ll still need a process that creates and ‘owns’ the ETS table as well as the bucket pool, but you’re right that the lookup and delegation does not need to happen in the Ratelimit process.
That would also eliminate the need for “forwarding” the call to the other processes.
Thanks!