mayank20
How does Oban Pro burst mode allocate slots across partitions and priorities?
How does Oban Pro burst mode allocate slots across partitions and priorities?
I’m trying to understand how burst: true behaves with partitioned global limits in Oban Pro.
For example, with a queue like:
queue1: [
local_limit: 5,
global_limit: [
allowed: 1,
burst: true,
partition: [args: :organization_id]
]
]
My understanding is:
-
Each partition gets 1 guaranteed slot (
allowed: 1) -
With
burst: true, partitions can temporarily use additional available slots up to the queue’s available concurrency (local_limit: 5in this case)
However, I’m confused about how burst slots are actually distributed.
Scenario 1 — Two active partitions
If there are jobs from 2 partitions only, I expected burst mode to greedily utilize all available slots.
But instead of something like:
partition_a = 3
partition_b = 2
I’m seeing behavior closer to:
partition_a = 2
partition_b = 2
1 slot idle
Why is the remaining slot left unused?
Scenario 2 — Three active partitions
Similarly, with 3 partitions active, I expected burst to continue filling remaining capacity, but it seems closer to:
partition_a = 1
partition_b = 1
partition_c = 1
2 slots idle
instead of using the full queue capacity.
Does burst mode not greedily allocate all available slots?
Scenario 3 — Priority interaction
I also noticed that priority doesn’t seem to affect burst allocation the way I expected.
Suppose:
-
Partition A jobs have priority
0 -
Partition B jobs have priority
0 -
Partition C jobs have priority
1
I expected all burst capacity to be consumed by priority 0 jobs first before any priority 1 jobs ran.
Instead, it seems like slots are distributed across all partitions regardless of priority, e.g.:
A = 1
B = 1
C = 1
Why does burst allocation behave this way?
How exactly does Oban Pro decide:
-
Whether additional burst slots should be used
-
Which partition receives them
-
Whether priorities are considered before burst distribution
-
Why queue capacity may remain idle even when runnable jobs exist
First Post!
sorentwo
It’s hard to answer definitively without knowing which Pro version you’re using, but I’ll answer based on how v1.7 works.
Burst here means “let each partition exceed the global allowed limit,” not “fill every slot”. The calculation favors fairness, so additional capacity isn’t given to a random partition. That’s why you’ll see a “2/2” split rather than “3/2”.
All currently-active partitions, where active is a briefly cached (3s) list of partitions with available jobs. Leftover capacity is discarded, not redistributed, in the interest of fairness.
Priorities are only honored within each partition’s query, it doesn’t apply priority between partitions. That’s semantically because of how the query and the underlying index are structured. Applying priority before the partitioning would require an additional pre-fetch query, which would double the work. A hybrid may be possible that gets closer to the priority behavior you expect.
This may be down to available partition caching. If there are jobs spread across multiple partitions, it may take several seconds before a job is in a visible partition.
Last Post!
mayank20
Popular in Questions
Other popular topics
Latest Oban Threads
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security









