Basically, I want to have the ability to create a job that will run periodically based on some cron expression.
AFAIK, Oban only supports defining cron per queue, not per job.
What I though about doing is creating a hook for after_process/2 where I can fetch the cron expression, calculate the next run using crontab lib and add a new job to the DB using scheduled_at option.
Is this the only option or is there another, better one?
Also, is there some corner case that I need to be aware if I go with the after_process/2 route?
You define cron jobs by worker, not by queue. The thing that’s inserted, worker + state, is the job. It’s fine to define mutliple cron entries for the same worker as well: Periodic Jobs — Oban v2.20.3
I can’t use periodic jobs because I need the cron expression to be dynamic, so I don’t know it before hand.
I guess the solution is to use recursive jobs as you mentioned. But I have a quick question about it. In the link, the new recursive job is added inside the perform function, I guess the idea is to guarantee that I can’t finish a job without inserting the new one, so there is no time window where the job finishes but the new job is not yet added to the DB. Basically to avoid scenarios where, let’s say, between the job finishing and inserting the new one, the server goes down and now the new job is never inserted in the DB but the old one is completed.
Do I have the same guarantee when inserting the job inside the after_process/2 function? I’m asking because I need to insert a new job even if the job failed for some reason, so I want to avoid having to handle errors/exceptions inside the perform function directly if possible since that can be fragile/error-prone.
Is there a common minimum period that you’re expecting in the crontab entries?
For instance, maybe they are intended to run at most once an hour.
Then you could have a standard-style cron-scheduled worker that runs every hour, and then checks the dynamic setting for if it should do anything THIS hour and no-ops otherwise.
That would avoid any worries about somehow “breaking the chain” that could arise with the “job schedules the next one” approach.
It can be any time that a cron expression can allow.
Basically I want my users to be able to schedule cron jobs in my system, so I don’t have control on when they would be triggered and why I wanted something dynamic.
For now, what I’m considering doing is having a cron_jobs table where I store the jobs with a field next_run_at and the cron expression.
Then, I have a normal oban job that runs every minute and check in cron_jobs for rows where next_run_at dates are older than the current time, then I fetch them, create oban jobs for each and update the next_run_at to be the next run based on the cron expression.
No, you don’t. An exception within after_process/3 won’t change the success of the original process/1 call. It could be catastrophic if it did, because a bug in a global after_process hook could make all of the jobs in a system fail.
