Fl4m3Ph03n1x
Background
I have a small terminal application that does HTTP requests to a website. However, I am making too many requests per second and I am drowning the receiver.
Solution
The obvious solution is to throttle the rate at which I perform requests.
Since I am using HTTPoison I thought about reading the docs to see if this was a feature, but I didn’t find anything.
The other solution is to find a community library that does this for me. This library would have to not use any external tools (like redis) and throttle out-going requests. After some investigation I didn’t really find anything in the community.
Surely this is something out there and I missed it.
Questions
What libraries do you use when you need to throttle out-going requests?
Trending in Questions
Hi all,
I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
I’m very new to Elixir, so sorry if this is a silly question:
I am wondering what happens to the mailbox of a process when the machine i...
New
How do I tell Hologram not to escape HTML when substituting into templates?
{Textile.render(@product.long_description)}
escapes the HTM...
New
Background
When trying to execute mix release on a Windows 11 machine for a Phoenix project I get the following error:
* assembling mark...
New
I’d like to create a file with 600 permissions (rw-------), but there’s a race condition if I create it and then change the permissions. ...
New
Is there a way to import record definitions in the iex shell for pretty printing (similar to the rr command in the erl shell)? This woul...
New
When I use Mix.install with a git dependency, using git@github.com:... how do I change the SSH identity used?
I’ve got two Github accoun...
New
Other Trending Topics
Hey there,
I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first.
...
New
Hi everyone :waving_hand:
I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #elixirconf-us
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #elixirconf-eu
- #processes
- #security
- #api
- #forms
- #performance











Showing Posts 1 to 5- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
lalo2302
There might be other libraries around which can handle this, but you can do this hands on and should not be very difficult.
Fl4m3Ph03n1x
Those are fair solutions, but if I wanted to do this myself I would have used an ETS table to queue up the requests as and then make batch requests to the target website.
I am just trying to avoid the workload.
cmkarlsson
There is an erlang library for load regulation that I have used in the past.
https://github.com/uwiger/jobs
At the simplest case you can start a job queue which will only allow a specific amount of requests per second. It doesn’t use any external dependencies.
At its simplest:
Runs the jobs at a rate of 10 per second at most
akoutmos
I actually wrote a blog post about this exact problem and go through the process of implementing token bucket and leaky bucket rate limiters using a simple GenServer and a Task.Supervisor: Easy and Robust Rate Limiting in Elixir-Alex Koutmos | Engineering Blog
I’d imagine that you want to persist your queue in the case that the application goes down. You can add that in pretty easily to the code snippets I walk through by implementing the
terminate/2callback.Hope that helps!
Fl4m3Ph03n1x
As a final remark to this thread, I just wanted to say I eventually went with the @cmkarlsson’s suggestion and used the jobs library.
However, I also read @akoutmos’s blog, and I can say that although it is not a reading for beginners, I very much enjoyed it and I can thoroughly recommend it to anyone wanting to learn more about throttlers.