Scheduling jobs with unique contrains

Special Sections Oban
,
1

Hey there!

I’m trying to understand how uniqueness works in Oban. I have the following worker:

defmodule ReminderWorker do
  use Oban.Worker,
    queue: :reminders,
    max_attempts: 3,

    unique: [
      period: 60,
      timestamp: :scheduled_at,
      states: [:scheduled, :executing, :retryable],
      fields: [:args, :worker],
      keys: [:id]
    ]
....

I already have a scheduled job for that worker:

  id  |   state   |                      args                      |    scheduled_at     |        inserted_at
------+-----------+------------------------------------------------+---------------------+----------------------------
 2171 | scheduled | {"id": "76014476-34e1-4221-bbeb-9414a2876d3e"} | 2024-05-30 18:19:00 | 2024-05-29 21:04:53.181529

If I try to schedule another job for the next day, I’m getting a conflict, which I understand should not happen because the scheduled_at date is a day off.

%{id: "76014476-34e1-4221-bbeb-9414a2876d3e"}
|> ReminderWorker.new(scheduled_at:  ~U[2024-05-31 18:00:00Z])
|> Oban.insert()

This is the result:

#Ecto.Changeset<
  action: nil,
  changes: %{
    args: %{id: "76014476-34e1-4221-bbeb-9414a2876d3e"},
    state: "scheduled",
    max_attempts: 3,
    queue: "reminders",
    worker: "ReminderWorker",
    unique: %{
      timestamp: :scheduled_at,
      keys: [:id],
      period: 60,
      fields: [:args, :worker],
      states: [:scheduled, :executing, :retryable]
    },
    scheduled_at: ~U[2024-05-31 18:00:00.000000Z]
  },
  errors: [],
  data: #Oban.Job<>,
  valid?: true
>
iex(3)> |> Oban.insert()
{:ok,
 %Oban.Job{
   __meta__: #Ecto.Schema.Metadata<:loaded, "public", "oban_jobs">,
   id: 2171,
   state: "scheduled",
   queue: "reminders",
   worker: "ReminderWorker",
   args: %{"id" => "76014476-34e1-4221-bbeb-9414a2876d3e"},
   meta: %{},
   tags: [],
   errors: [],
   attempt: 0,
   attempted_by: nil,
   max_attempts: 3,
   priority: 0,
   attempted_at: nil,
   cancelled_at: nil,
   completed_at: nil,
   discarded_at: nil,
   inserted_at: ~U[2024-05-29 21:04:53.181529Z],
   scheduled_at: ~U[2024-05-30 18:19:00.000000Z],
   conf: nil,
   conflict?: true,
   replace: nil,
   unique: nil,
   unsaved_error: nil
 }}

My understanding is that the unique constraint should exclude jobs with the same scheduled_at date, not with different ones. What am I missing?

Thanks a lot!

2

It looks correct to me. I setup a quick project with your config and could not reproduce the issue.

One thing you can try is setting the oban config level to debug.

config :my_app, Oban,
  engine: Oban.Engines.Basic,
  queues: [default: 10],
  log: :debug,
  repo: ObanTest.Repo

You can then see the SQL oban is using to check for a conflicting job. Check this matches your expectations (especially scheduled_at and states)

1 Like
3

The unique query checks for a job with a datetime greater than or equal to the current time, not the new scheduled time. In your case, a job exists that’s well in the future. Changing the comparison could be more intuitive in situations like this, but I’m afraid it would be breaking for many others.

I suggest changing the args and unique format to better suite your use case. The timestamp, fields, and states aren’t necessary. Switch to an infinite period and add another key:

unique: [period: :infinity, keys: [:id, :date]]

Then schedule jobs with the id as before, and the date you want them to run:

new(%{id: "some-uuid", date: ~D[2024-05-30]})
2 Likes
4

I was just debugging using what @msmithstubbs suggested and I noticed this:

That explains why it was failing. And yes, changing the comparison would make it much easier, but I understand the consequences of it.

I’ll follow your suggestion. Thanks a lot for your quick answer !

1 Like