How to rewrite scheduled Oban jobs into a new queue, worker, and args?

Special Sections Oban
1

I have a bunch of scheduled jobs in queue A. The args have a certain “shape”.

I need to move those jobs into queue B. Args for jobs in queue B are a different shape, where the args from A will nest into each job in queue B so that the B Worker can consume them correctly.

I’m planning on writing this as an in-place SQL data migration, where I’ll do my own SELECT ... FOR UPDATE SKIP LOCKED to rewrite the rows with new args, a new worker, and a new queue. I’d like to keep the job IDs, priority, meta, etc. the same between them, so the only things that would change would be the queue, worker, and args.

Is this reasonable? What “gotchas” should I be aware of when attempting something like this?

1 Like
2

This is really about the worker and not the queue, as the queue doesn’t care what jobs are in it, but the goal makes sense :slightly_smiling_face:

Unless you have a lot of jobs that may be overkill. Can you pause the queue, start a transaction, update the job records, then resume the queue.

That’s totally reasonable. Make sure the args matches the shape expected by the new worker and it will be fine.

3 Likes
3

Thanks for the help!

It’s good to know that we could do the migration this way. We do have a lot of jobs to rewrite, and there will be some jobs that need to execute in a timely fashion that will be out of scope for this migration. The transform from A to B involves potentially talking to some separate services and applying some business logic to translate the jobs, but it’s good to know that just updating the records in place to have a new worker (and probably a new queue as well) should do the trick.

Thank you!

1 Like