Do chains of workers span queues?

Special Sections Oban
,
1

If I have a worker which is chained and the jobs are enqueued on to multiple queues does the chain span those queues?

So let’s say I have this:

defmodule Kafka.Inbox.Worker do
  use Oban.Pro.Workers.Chain,
    by: [:worker]

  def process(job), do: IO.inspect(job)
end


[
Kafka.Inbox.Worker.new(queue: "red"),
Kafka.Inbox.Worker.new(queue: "blue"),
] |> Oban.insert_all()

Would the job on the “red” queue wait for the job on the “blue” queue?

(I’m assuming that insert_all will enqueue the first job first in the list first).

Or are the chains independent per queue.

Thanks, Kris.

2

No. Chains are always per queue. You can’t have a chain that crosses queues.

Why? Chaining always includes the queue field for performance and correctness. This means that specifying by: :worker is equivalent to by: [:queue, :worker], and by: [args: :account_id] is equivalent to by: [:queue, args: :account_id]. Jobs in different queues will never be part of the same chain. Ideally, this is to ensure better performance and prevent unexpected dependencies across queues.

2 Likes
3

Perfect, thanks!

I was putting “queue” in the job payload and chaining on that arg, so I don’t need to do that.

Might be worth adding a note to the doc page about this :+1:

4

You are absolutely correct. Getting on that!

Thank you!