D4no0
Barrier synchronization for oban jobs
I am using oban to create a group of tasks, all of these tasks are linked to the same parent. After all the tasks are complete, I want to add additional information to that parent that the tasks have completed, so a different status can be rendered to the user.
Additionally to this, there is no guarantee on the order and concurrency of executed tasks, some might fail and require a retry, the server might be restarted when a new deploy is done (hence why oban is used in the first place).
I was thinking on a few approaches:
- Exectue a query to check if other jobs are complete at the end of every job, if yes update the status. I do have my fears about this not working correctly in concurrent setting, maybe someone knows more on this topic;
- Have a background worker that will periodically check number of jobs completed and update the status, this of course implies some potential delays and queries executed, but for my project it is a viable solution;
- Monitor the tasks with a process, re-synchronize the process based on data from the database when the server is restarted.
If you have any better idea it would be great to hear a different perspective.
Most Liked
lamxw2
That looks like a good idea, however I have a concern where I tried using Batch in the past and dropped it because we handle our own retries. We wanted to retry jobs based on specific errors, not all, so we insert each retry as a new job. I believe I may run into the same issue here with Workflow.
The other issue I had with Batches (which according to the docs is the same for Workflows) Oban.Pro.Worker — Oban Pro v1.5.0-rc.9
If your process function returns an
{:ok, value}tuple, it is recorded. Any other value, i.e. an plain:ok, error, or snooze, is ignored.
We needed to record errors as well.
sorentwo
Errors are recorded in the errors array on the job anyhow. So if you return {:error, error}, or raise, or crash, that’s listed in errors, but it’s not duplicated as a recorded value.









