I do tend to think that if something comes up multiple times it would probably benefit from a blessed solution.
bennelsonweiss
sorentwo
Oban Core Team
Oban v1.1.0 is out with some minor features and a ton of reliability improvements prompted by edge cases from real world usage. Thanks to everybody that reported the issues and helped in proving the solutions 
From the CHANGELOG:
Fixed
-
[Oban.Crontab] Allow any number of spaces and/or tabs in cron expressions.
-
[Oban.Pruner] Prevent deadlocks while pruning by ensuring that only a single node can prune at any given time. (Thanks @bgentry)
-
[Oban.Queue.Executor] Improve handling of nested/linked process failures. Previously if a job spawned a process that crashed (i.e. through Task.async) then the job was left stuck in an executing state.
Added
-
[Oban.Worker] Add
timeout/1callback for limiting how long a job may execute. The default value is:infinity, which allows a job to run indefinitely and mirrors the previous behavior. (Thanks to @benwilson512 who did the initial implementation) -
[Oban.Telemetry] Add
:errorand:stacktotrip_circuitevent metadata. -
[Oban.Queue.Executor] Retry success/failure database updates after a job finishes.
On occasion something may happen to the database connection and updating a job’s state would fail. Now we retry writing with a linear backoff to prevent the job from getting stuck in an executing state. (Thanks to @coladarci)
Changed
-
[Oban.Worker] Tighten the spec for
perform/2. Now workers are expected to return only:ok,{:ok, result}or{:error, reason}. A warning is logged if any other value is returned—for high throughput systems this may cause performance issues and you should update your worker’s return values.Returning a success tuple is supported for testing purposes and backward compatibility, while returning
:okon success if preferred. (Thanks to @sasajuric)
13
smolcatgirl
Hi,
I have a question. I use a API that bans or rate limits its users if they do more than one request in 4 seconds. Is there any to make some kinda interval so that if I have some jobs related to this api and some not, then the there will be always be 4 seconds between each job with calls to this api and the worker could run other jobs meanwhile. If there are no other jobs then the worker could just wait until 4 seconds has passed. Any thoughts on this?
Thanks for your time! 
DiodonHystrix
Simplest solution could be to define a queue with concurrency set to 1 for that jobs and sleep necessary time at the end of each job.
If it’s not possible, I’d use GenServer to throttle the requests.
smolcatgirl
Simplest solution could be to define a queue with concurrency set to 1 for that jobs and sleep necessary time at the end of each job.
This is what I lean towards right now but I find it ugly.
sorentwo
Oban Core Team
You can model basic rate limiting by conditionally returning an error tuple:
defmodule MyApp.LimitedWorker do
# You may want to use a longer period
use Oban.Worker, unique: [period: 4]
@impl Oban.Worker
def perform(%{"user_id" => user_id}, _job) do
if MyApp.Accounts.over_rate_limit?(user_id) do
{:error, :over_rate_limit}
else
do_whatever_this_worker_does()
:ok
end
end
# Retry every four seconds, you'd probably want this to be smarter.
@impl Oban.Worker
def backoff(_attempt), do: 4
end
That version will retry the job later after the rate limit has cooled down. Note that you’ll get error reports for each retry, so you may want to ignore the :over_rate_limit error in your error reporter.
smolcatgirl
Thanks. To me Oban seems really great but also really complex and I have a hard time grasping it. Is there some example codebase to look at to understand better?
sasajuric
Author of Elixir In Action
You could use jobs for that, it supports rate limited queues. Note that job` is not about persistence, so you might still combine it with oban. Basically, in each worker perform you ask your jobs-based rate limited queue for permission. Since jobs is in-memory, a system restart might cause an occasional rate offense (e.g. you execute the job, then the system restarts, and you end up executing another job in the interval of 4 seconds).
jc00ke
Are there specific things you find confusing that we could help you, and others, understand? We could add these topics to a knowledge base/wiki like Sidekiq’s wiki.
In terms of the rate limiter part, I’ve used ExRated in the past (though I wish it were named something different) and ExHammer also looks very nice. Those you could use for the
part. You would still likely have to “fail” the job so Oban will retry later.
anthonator
Is there any sort of way to make assert_enqueued act like assert_receive? Or is there some sort of monitor I can subscribe a test process to so I can get notified when a job gets enqueued before using assert_enqueued? There are times I want to test a job gets enqueued as part of an async process.