Large number of queues in Oban

[iampeter]

I am wondering what would the (performance?) implications be of having a large number of queues in Oban, let’s say 100000.

What I want to achieve is preserving order of jobs per one entity, so let’s assume I have 100000 purchases in a store, I would like to have one queue per one purchase so that jobs associated with each purchase are processed FIFO for that purchase.

Or is there a better way to preserve job processing sequence? I guess it cannot be achieved by unique jobs, because I would be able to schedule more than one at a time?

Thanks in advance,
PG

[benwilson512]

@iampeter

Or is there a better way to preserve job processing sequence?

Oban Pro with partitioning and a global limit of 1 per partition will do exactly this without the overhead of 10k queues.

If you want to guarantee not only one per item but also a strict sequence you can use the chain worker with those same partition settings.

[sorentwo]

@iampeter

Or is there a better way to preserve job processing sequence? I guess it cannot be achieved by unique jobs, because I would be able to schedule more than one at a time?

@benwilson512’s answer is correct—chain’s are designed for this exact use case Oban.Pro.Workers.Chain — Oban Pro v1.4.10

[iampeter]

Thanks for your answers.