Oban: Skipping a run of a job

Hi!

I’m a happy user of Oban Pro, which have simplified a lot of things for me. However, I have found a use case which I’m not sure if Oban supports.

I’m building an e-commerce system for tickets. After the customer pays an order we send an API request to another system that confirms an earlier “temporary” order. After the order is confirmed, we need to do another request to get the actual ticket. However, this ticket is not available immediately. Sometimes it’s available after 1 second, sometimes it takes a minute, in some extreme cases it takes an hour.

I thought I should set up an Oban worker that attempted this ticket-fetching API request, that had a backoff strategy of something like 10, 10, 10, 10, 10, 20, 40, … So in other words, constant 10s for the first five retries, then exponential. This works, but to be able to get it to re-run, I have to return an error tuple. This means that every time this happens, I get an error logged, for a case that is actually expected.

I know I can snooze a job, but then I’d have to re-implement the backoff for the snooze case as well as the backoff/1 callback.

I can of course filter this error in the error handling part, but that doesn’t seem a very good solution, leaking details about the ticket fetching to the generic error handling.

Now that I write this, I realize that backoff/1 is a function, so maybe I can just call it to get the value for the snooze parameter. But in that case, should I just send the job as is, or do I need to do something with the attempts value?

Another option is to have the possibility to return :skip or something similar from perform/1 that just queues the job again, with backoff.

Again, thanks so much for Oban. It has made things much easier!

Hello,

When you return {:snooze, time_in_seconds} from your perform/1, it does increase the attempts numbers, (but also the max_attempts number so you are still able to fail X times with actual errors). You can then decide how much to snooze according to the attempts number.

This seems actually harder to achieve with oban pro:

Oban Pro’s Smart Engine rolls back the attempt and preserves the original max_attempts in order to differentiate between “real” attempts and snoozes, which keeps backoff calculation accurate.

Without attempt correction you may need a solution that compensates for snoozing, such as the example below:

But maybe you can compute the backoff not from the attempts but from the date of your order? If it’s less than a minute old, return 10s, if it’s less than 5 min, return 20s, otherwise return 1min…

That should be a last resort, there are ways to track the attempts and calculate a custom backoff from your worker module.

The number of snoozes is also recorded in the job’s meta, so you could preserve the original backoff/1 functionality:

def process(job) do
  if polling_thing_finished do
    ...
  else
    {:snooze, backoff(job)}
  end
end

def backoff(job) do
  total_attempts = job.attempt + Map.get(job.meta, "snoozed", 0)

  if total_attempts < 6 do
    10
  else
    super(%{job | attempt: total_attempts})
  end
end

There’s also the private, but well documented, Backoff.exponential function that you could call directly if you’re feeling adventurous.

1 Like

Perfect, snoozing seems to be the way to go imho.

Oh, that’s a very clean solution. Thanks!