That is the most effective and efficient way to do it, though not a trivial undertaking. A snooze + priority approach could work if you can tolerate the churn and inefficiency.
This use case keeps coming up and I’m definitely looking to support it in Pro 
Yeah, we’re getting the subscription (in the process) but it looks fairly straightforward with the way Oban is architectured.
Let me reanimate this thread with extra questions @sorentwo , at the moment of legal nature.
So we have migrated from Exq to Oban very smoothly, and indeed we’re using Pro + Web UI. We’re already using SmartEngine that comes with Pro, and we’re using it’s unique features that are not in free version, i.e. rate_limit and also global_limit.
Now, here’s the legal question: we would like to add the rate limit per tenant functionality one of our queues, but keep the pro’s unique rate_limit / global limit features we already use for some other queues.
Can I just grab and modify the smart_engine.ex from pro distribution that comes into deps/ or would that be a crime of century?
Edit: I would also be happy with using different engines per different queues. Not sure how to do it or if it’s possible either, however.
Not the crime of the century, but also not something I can endorse.
This is something we’d love to provide to everybody that uses Pro. If you’re willing to share more details of your use case I’d love to talk through an API and implement it in Pro directly. You’re paying for Pro for a reason! Send a message if you’re interested
Well, I would actually prefer if you implemented it yourself, so I don’t have to so if we can avoid crime as well and work for me that’d be ideal.
What we have is a multi-tenant system, where each tenant can schedule a number of background jobs. We want to prevent one, hugely intense tenant, from taking up all resources of the system, by queuing up thousands of background jobs.
These background jobs as well communicate with external API, which is rate limitted, and each tenant is using own API key meaning if they schedule and execute thousands of jobs at the same time they will also exhaust their API limits.
The solution to both of our problems would be to have rate limiting per queue, per tenant, or have a set of queues per tenant and rate limit them separately.
we currently don’t use anything like PostgreSQL schemas to separate tenants. Each tenant has ID, and their associated records all have tenant_id set on them.
I was thinking about having tenant_id as argument to each task scheduled as well. Then, I would fetch the jobs from tenants that are not exceeding the limits. But I am open to different solution that achieves the same goal.
I believe I have a suitable API between our prior discussion, some outside requests, and the details you’ve provided here. It should be flexible enough to support a few different use cases, introspectable enough to figure out what’s going on in the engine, and fast enough to keep jobs processing smoothly.
I’ll spare the implementation details and focus purely on the SmartEngine API changes.
The feature is tentatively called “Partitioned Rate Limiting,” which will build off the existing rate-limit feature. The configuration spec would look like this:
partition: [fields: [:worker | :args], keys: [atom()]]
Translating that into your tenant_id use case:
rate_limit: [
allowed: 60,
period: :timer.seconds(60),
partition: [fields: [:args], keys: [:tenant_id]]
]
Alternatively, if somebody wanted to rate limit purely by the worker:
rate_limit: [
allowed: 60,
period: :timer.seconds(60),
partition: [fields: [:worker]]
]
The engine will then track and apply rate-limiting per-partition.
Will that work for you?
Working on it now that some major 
is finished. Of course, there’ve been many changes behind the scenes to improve queue option validation and serialization, but the groundwork is there.
As soon as something is ready for testing, I’ll announce it here 
