I develop bioinformatics pipelines as my job. Sun GridEngine(SGE) is used to dispatch computation intensive jobs of a pipeline to a cluster of compute nodes. Currently, I am trying to build a mini-tool that is able to detect dna sequencing is done, demultiplex sequencing data, launch an analytical pipeline, and generate a report. All these are associated with background job processing, so I want to use oban as the building block.
I hope oban is able to handle submitting a job via qsub as a kind of background job processing, and poll SGE to determine if a job is completed.
1 Like
Sure, you can do that with Oban, just as you could with a task or any async process. There isn’t any limit on how long the job runs.
If you’ll only poll for a few seconds then use a recursive loop with a small sleep period. If you need to poll for minutes then you can snooze the job and check again after a few seconds.
3 Likes
After I check the Oban doc, I find that perform/1 in a worker returning {:snooze, seconds} can do this. I wonder that what telemetry event will be issued when snoozing the job. I guess it is definitely not a job completed event, right?
The telemetry event’s state is :snoozed. You can see the event details in the Oban.Telemetry docs.
