I remember in the Rails days using Sidekiq but then also lugging around Clockwork just for scheduling. When glancing your readme one of the first things I looked for was repeated scheduled tasks.
I wonder what others think about putting oban_cron into the main repo.
Informally I’ll consider “hearts” on this post to be in favor of including it directly in Oban. If anybody has arguments against inclusion please post!
reads post, hearts it as confirmation
Lol, but really, it’s not hard to not use it and has no overhead if not used if it’s not needed, and is very specific to oban anyway, so…
I think that it would be a great addition to Oban.
The current version of Quantum is unstable when clustering in certain cases. We’ve experienced some of them in the company I work for which caused some undesirable effects such as tasks not running or running once per each node in the cluster.
This is acknowledged in Quantum and they are working in a solution.
Oban has been rock solid and reliable since we started using it in the project. So it could be a wonderful alternative.
@cnck1387 @belaustegui I merged Periodic (CRON) support into Oban today. Please check it out, it will be included in v0.11 sometime next week.
Thanks for the encouragement!
Nice work. Looking forward to replacing Quantum with this. After just glancing at your commit, that makes me really happy to see. I had nothing against Quantum but being able to drop a whole library with a ton of code for a few dozen lines of code is a huge win.
One question. In your new docs you put “Jobs are considered unique for most of each minute”. What type of race conditions or edge cases should we be aware of to hit the points where it might not be unique?
Jobs are marked as unique for 59 seconds, not 60 seconds. There has to be some wiggle room between the unique period and the next enqueue cycle. That leaves a one second window where a theoretical double-enqueue is possible. That situation would only happen if you can restart your node fast enough that it enqueued at the start of one second, booted up and enqueued at the end of the same second.
If I’m understanding correctly, this supports running a job on a single node in a cluster periodically.
It might be beyond the scope of Oban, but is there a way to run a job on every node in a cluster periodically? I’ve used Quantum that way for things like refreshing credentials periodically.
Right. That’s the goal.
I hadn’t considered that use case. It would certainly be possible: eliminate the transaction lock and remove the unique period while scheduling.
You could kind of hack it with the current implementation by overriding the unique period within your job so that it’s very short:
use Oban.Job, queue: "scheduled", unique: [period: 1]
If the nodes start at different times that would work, but it wouldn’t be very reliable.
I’ll think about this use case a bit.
Oban v0.11.0 is published with a variety of bug fixes and the addition of CRON jobs. There is an optional migration that will prevent issues recording beats when job ids get into the 64bit range.
Anybody using the UI beta should upgrade, the oban_update notification change is essential to keeping stats updated.
From the CHANGELOG
Migration Optional (V6)
Job id’s greater than 2,147,483,647 (PG int limit) can’t be inserted into the running array on oban_beats. The array that Ecto defines uses int instead of bigint, which can’t store the larger integers. This migration changes the column type to bigint[], a locking operation that may take a few seconds.
crontab support for automatically enqueuing jobs on a fixed schedule. A combination of transactional locks and unique jobs prevents scheduling duplicate jobs.[Oban.Migrations] Add a comment when migrating oban_jobs to V5 and when rolling back down to V4.
[Oban.Query] Apply the configured log level to unique queries.
[Oban.Notifier] Prevent open connections from accumulating when the circuit is tripped during the connection phase. This change may leave notifications in a state where they aren’t listening to all channels.
oban_update notifications to subscribed processes.