Update job args inside the job

Special Sections Oban
, ,
1

In my system, I have a job that needs to do two steps, if the first step is done and the second step fails, the job should retry directly from the second step.

For example, let’s say my job needs to add a row to a table A. That would be step 1. After that, I want to add other rows to another table B that has a relationship with table A.

If I added the row to table A and then the job failed during inserting the other rows to table B, I don’t want it to create another row in table A during the retry, i want it to know the step 1 is already done, fetch the row and retry step 2.

One obvious way to do this is simply separate the steps into two jobs and insert the second job in the first at the end of step 1, but I can’t do this since I send the job ID to the client to keep checking the job status, meaning that when the job is running step 2, it will simply show the job as completed since step 2 is running in another job with another id.

What I think would be a plausible solution would be to update the args of the job itself when running step 1 adding the table A new row id to it, that way, if the job retry, it will already have that row id in the args and I would be able to fetch it and continue from step 2.

Is that possible to do?

1 Like
3

It is possible to update the args using the Oban.Job struct, Ecto.Changeset, and a Repo.update call, but there’s no blessed way to do it with Oban directly. Updating job args as the job is executing isn’t recommended and may cause issues in the future.

You’re better off inserting a new job and using something like a uuid to link the jobs rather than the job’s id.

3 Likes
4

Do you mind giving a little bit more details about your suggestion? I not sure I quite get what you meant with it.

Do you mean adding an arg to the job with a common uuid and then using that in a Ecto.Query.where to find that job instead of searching by the job id?

If so, wouldn’t that do a seqscan?

5

Yes, that’s what we’re suggesting.

That depends on how you construct the query. Here is an example that uses the index on args:

where(Oban.Job, [j], fragment("? @> ?", j.args, %{uuid: "some-uuid"}))

This could be an ideal case for a workflow. Of course, workflows are heavy on our minds as of late. :wink:

1 Like