Is there a way to set an arbitrary global limit while partitioning globally in Oban?

Special Sections Oban
1

Hello,

We currently have a queue that has the following configuration:

my_queue_config: [
         local_limit: 10,
         global_limit: [allowed: 3, partition: [args: [:tenant_id]]]
]

When re-deploying and during the release of a new version of our Elixir application, the number of nodes temporarily doubles. We end up in a situation where the number of executing jobs also doubles.

Let’s say we want a maximum of 15 concurrent job across the cluster.
Is there a way to set an arbitrary global limit while partitioning globally?

Thank you!

2

No, there isn’t a way to layer global limits like that currently. Once you have partitioning applied, it overrides the standard global limit. Something you can try is temporarily scaling down the local limit on the active node or pausing the queue on startup and resuming once the nodes are back to a stable number.

2 Likes