D4no0
I am using oban to create a group of tasks, all of these tasks are linked to the same parent. After all the tasks are complete, I want to add additional information to that parent that the tasks have completed, so a different status can be rendered to the user.
Additionally to this, there is no guarantee on the order and concurrency of executed tasks, some might fail and require a retry, the server might be restarted when a new deploy is done (hence why oban is used in the first place).
I was thinking on a few approaches:
- Exectue a query to check if other jobs are complete at the end of every job, if yes update the status. I do have my fears about this not working correctly in concurrent setting, maybe someone knows more on this topic;
- Have a background worker that will periodically check number of jobs completed and update the status, this of course implies some potential delays and queries executed, but for my project it is a viable solution;
- Monitor the tasks with a process, re-synchronize the process based on data from the database when the server is restarted.
If you have any better idea it would be great to hear a different perspective.
Where Next?
Trending in Questions
Other Trending Topics
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
Elixir, Erlang & messaging conferences from Code Sync. Next up: ElixirConf US (10–11 Sep) & Code BEAM Europe (21–22 Oct), Haarlem + online.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Master modern tools, write cleaner code, and tackle real-world challenges.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
dimitarvp
Maybe the managers aren’t looking to pay but Oban Pro seems to have what you’re looking for: https://oban.dev/docs/pro/1.2.2/Oban.Pro.Workers.Chunk.html
Though I’m not 100% sure. If you add DB updating logic at the end of each job in the chunk, and have another job sniffing the database and waiting for the data structure to say “all related sub-jobs have completed successfully”, then it will likely achieve what you need.
Though now thinking of it, that could be achieved with Oban’s free version as well.
D4no0
Sadly Oban pro is out of the question, there are no resources for such expenses, moreover it seems an overkill for such a simple feature.
So you are in favor of the solution 2? Makes sense as it would avoid having to deal with any race conditions.
herisson
hey, did you ended up with a working solution not requiring Oban pro ? That would help me a lot.
Thanks!
D4no0
I never got to that, as that project got frozen for the time being, however implementing the solution I outlined as number 2 should be more than good enough if you don’t care about this happening instantly.
This can be as easy as creating a genserver that will query the database every N seconds/minutes and update the status, should be no more than a couple of lines of code that is foolproof.
herisson
ok thanks i’ll try that !
mukulmajmudar
Your first approach can work too, as long as the “check and update” operation is synchronized among the many jobs.
lamxw2
Your fears are right. I used the 1st option and I am facing race condition issues. Came here looking for possible solutions too.
hubertlepicki
If you have Pro subscription, I believe this is the case for Workflow.Worker.
lamxw2
That looks like a good idea, however I have a concern where I tried using Batch in the past and dropped it because we handle our own retries. We wanted to retry jobs based on specific errors, not all, so we insert each retry as a new job. I believe I may run into the same issue here with Workflow.
The other issue I had with Batches (which according to the docs is the same for Workflows) Oban.Pro.Worker — Oban Pro v1.5.0-rc.9
We needed to record errors as well.
sorentwo
Errors are recorded in the
errorsarray on the job anyhow. So if you return{:error, error}, or raise, or crash, that’s listed inerrors, but it’s not duplicated as a recorded value.