Understanding slot allocation and fairness in partitioned queues

[mayank20]

Hi, I’m using Oban Pro v1.6.13 with a queue configured like this:

queue1: [
  local_limit: 5,
  global_limit: [
    allowed: 1,
    burst: true,
    partition: [args: :organization_id]
  ]
]

I’m trying to better understand how slot allocation/fairness works internally for partitioned queues.

In this setup, effectively there can be many active partitions (organization_id values), while only 5 execution slots are available because of local_limit: 5.

Suppose there are 10 active partitions, all holding runnable jobs.

My questions are:

1. How does Oban decide which 5 partitions receive slots first?

2. Is there any deterministic ordering or rotation between partitions?

3. Once those 5 partitions start processing, and one job completes, does Oban:

   • continue giving preference to already-running partitions, or

   • try to give other waiting partitions a chance? If yes, on what basis are these new partition selected?

Thanks!

[sorentwo]

@mayank20

My questions are:

1. How does Oban decide which 5 partitions receive slots first?
2. Is there any deterministic ordering or rotation between partitions?
3. Once those 5 partitions start processing, and one job completes, does Oban:

• continue giving preference to already-running partitions, or
• try to give other waiting partitions a chance? If yes, on what basis are these new partition selected?

Jobs in partitioned queues use the same priority rules as any other jobs (priority, then scheduled time). The ordering between partitions is deterministic, though it’s possible to have ties if there are enough jobs with the exact same scheduled_at timestamp.

There’s no preference for previously used partitions beyond the cache period, which is configurable.