Unique Workflows (no overlap per user)

Depending in the users amount, I would use a GenServer per user, saving in the state the jobs queue and a tick function wich take the first in the list, execute and update the state…

If you want to persist the state in case of crashes and you are not using a cluster, you can save the state in ETS… and load it again in the init callback.

If you are using a cluster, you can use Horde and recover the state with :continue flag.

I’m not sure if I would use Oban for this case…

it’s an Oban Workflow that I am trying to run uniquely

Yes! Indeed, there is. With Pro v1.6 you can set a global partition by workflow_name or some other unique attribute of that users workflow. (That’s really up to you). It’s more about concurrency and not uniqueness.

config :my_app, Oban,
  queues: [
    queue_name: [limit: 20, global_limit: [allowed: 1, partition: [meta: :workflow_name]]]
  ]

In this case, each node can run up to 20 jobs but only 1 of that particular workflow. It relies on you setting a distinct workflow name for each user.

Workflow.new(workflow_name: "some-workflow-#{user_id}")

Nice! Is 1.6 ready for prod use yet?

Just reading 1.6 docs now.

Can I use the new context in the partition?

e.g.

global_limit: [allowed: 1, partition: [meta: :workflow_name, context: :user_id]]

*edit: looks like no. workflow_name will work, though. thanks

Worker name or workflow name?

yeah workflow_name

We run it in production. It’s in rc for a while, at least a month. Totally up to you.

*edit: looks like no. workflow_name will work, though. thanks

Bummer. Is that because Oban doesn’t support partitioning by context keys yet?

The context is stored by itself, it isn’t stored with every job. That means it’s impossible to partition by anything within the context because it’s simply not available on every workflow job.