mayank20

mayank20

How does Oban Pro burst mode allocate slots across partitions and priorities?

How does Oban Pro burst mode allocate slots across partitions and priorities?

I’m trying to understand how burst: true behaves with partitioned global limits in Oban Pro.

For example, with a queue like:

queue1: [
  local_limit: 5,
  global_limit: [
    allowed: 1,
    burst: true,
    partition: [args: :organization_id]
  ]
]

My understanding is:

  • Each partition gets 1 guaranteed slot (allowed: 1)

  • With burst: true, partitions can temporarily use additional available slots up to the queue’s available concurrency (local_limit: 5 in this case)

However, I’m confused about how burst slots are actually distributed.

Scenario 1 — Two active partitions

If there are jobs from 2 partitions only, I expected burst mode to greedily utilize all available slots.

But instead of something like:

partition_a = 3
partition_b = 2

I’m seeing behavior closer to:

partition_a = 2
partition_b = 2
1 slot idle

Why is the remaining slot left unused?

Scenario 2 — Three active partitions

Similarly, with 3 partitions active, I expected burst to continue filling remaining capacity, but it seems closer to:

partition_a = 1
partition_b = 1
partition_c = 1
2 slots idle

instead of using the full queue capacity.

Does burst mode not greedily allocate all available slots?

Scenario 3 — Priority interaction

I also noticed that priority doesn’t seem to affect burst allocation the way I expected.

Suppose:

  • Partition A jobs have priority 0

  • Partition B jobs have priority 0

  • Partition C jobs have priority 1

I expected all burst capacity to be consumed by priority 0 jobs first before any priority 1 jobs ran.

Instead, it seems like slots are distributed across all partitions regardless of priority, e.g.:

A = 1
B = 1
C = 1

Why does burst allocation behave this way?

How exactly does Oban Pro decide:

  1. Whether additional burst slots should be used

  2. Which partition receives them

  3. Whether priorities are considered before burst distribution

  4. Why queue capacity may remain idle even when runnable jobs exist

First Post!

sorentwo

sorentwo

Oban Core Team

It’s hard to answer definitively without knowing which Pro version you’re using, but I’ll answer based on how v1.7 works.

Burst here means “let each partition exceed the global allowed limit,” not “fill every slot”. The calculation favors fairness, so additional capacity isn’t given to a random partition. That’s why you’ll see a “2/2” split rather than “3/2”.

All currently-active partitions, where active is a briefly cached (3s) list of partitions with available jobs. Leftover capacity is discarded, not redistributed, in the interest of fairness.

Priorities are only honored within each partition’s query, it doesn’t apply priority between partitions. That’s semantically because of how the query and the underlying index are structured. Applying priority before the partitioning would require an additional pre-fetch query, which would double the work. A hybrid may be possible that gets closer to the priority behavior you expect.

This may be down to available partition caching. If there are jobs spread across multiple partitions, it may take several seconds before a job is in a visible partition.

Last Post!

mayank20

mayank20

Great, will be waiting for it.

Where Next?

Popular in Questions Top

vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement