Oban Chunk worker jobs stuck in executing state

Special Sections Oban
1

Hey all!

We have some Oban batch workers that look like this:

  use Oban.Pro.Workers.Chunk,
    queue: :explorer_shipments,
    size: 1000,
    timeout: 1000,
    unique: [
      keys: [:shipment_id],
      states: [:available, :retryable, :scheduled]
    ]

and then the process function has

thing1 = Task.async(fn -> compute_thing1(args) end)
thing2 = Task.async(fn -> compute_thing2(args) end)

_ = Task.await_many([thing1, thing2], 30_000)

Periodically, one of those two tasks will crash. When this happens, the process related to the job definitely crashes, but the Oban jobs related to that process (there are several since it’s a chunk worker) all seem to be left in the executing state indefinitely.

Is this some sort of configuration issue on our end? A bug in Oban? Any insights would be appreciated.

1 Like
2

The process exit bypasses the chunk job’s task and hits the underlying producer process. That has no idea what a chunk is, and doesn’t clean up those errored jobs.

There’s no misconfiguration or problem with what you’re doing. We’ll have to fix it :slightly_frowning_face:

2 Likes