Potential Oban Pro bug testing workflows with cancelled jobs

Special Sections Oban
, ,
1

We are currently upgrading to the latest Oban and Oban Pro dependencies and are seeing an issue testing workflows that cancel jobs.

This is a simplified version of the test we are running. The idea is that something happens in CancelWorker that causes us to cancel the rest of the workflow. RecordArgsWorker is a test helper job we use.

workflow_id = "test-workflow"

Workflow.new(workflow_id: workflow_id)
|> Workflow.add(:a, RecordArgsWorker.new(%{}))
|> Workflow.add(
  :b,
  CancelWorker.new(%{}),
  deps: [:a]
)
|> Workflow.add(:c, RecordArgsWorker.new(%{}), deps: [:b])

assert %{completed: 1, cancelled: 2} = run_workflow(workflow)

This test worked with Oban 2.18.3 and Oban Pro 1.5.0-rc.7. It fails after we upgrade to Oban 2.19.1 and Oban Pro 1.5.0 with the following exception.

** (Ecto.Query.CastError) deps/oban_pro/lib/oban/pro/engines/smart.ex:1393: value `"draining"` cannot be dumped to type :binary_id in query:

     from p0 in Oban.Pro.Producer,
       where: p0.uuid == ^"draining",
       where: fragment("?->'global_limit' ? 'tracked'", p0.meta),
       select: p0

     stacktrace:
       (elixir 1.17.3) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
       (elixir 1.17.3) lib/enum.ex:1829: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
       (elixir 1.17.3) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
       (ecto 3.12.5) lib/ecto/repo/queryable.ex:214: Ecto.Repo.Queryable.execute/4
       (ecto 3.12.5) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
       (ecto 3.12.5) lib/ecto/repo/queryable.ex:154: Ecto.Repo.Queryable.one/3
       (oban_pro 1.5.0) lib/oban/pro/engines/smart.ex:1395: anonymous fn/2 in Oban.Pro.Engines.Smart.track_cancelled_jobs/2
       (elixir 1.17.3) lib/enum.ex:992: anonymous fn/3 in Enum.each/2
       (stdlib 6.1.2) maps.erl:860: :maps.fold_1/4
       (elixir 1.17.3) lib/enum.ex:2543: Enum.each/2

Digging through code in my deps directory, I see how the invalid UUID gets there when draining jobs, but I’m not sure what the fix is. And I don’t want to be posting Oban Pro code here. :sweat_smile:

Is this a known issue? We can wait on the upgrade if necessary until this is fixed or we find a workaround.

2

The “fix” to isolate draining workflows was added in a subsequent RC. It makes sense that it would fail after upgrading.

Much appreciated! We’ll take care of the fix :slightly_smiling_face:

It wasn’t a known issue, but it is now. There’s no real workaround and you’ll need to hold off on an upgrade until we get a patch out.

2 Likes
3

This issue has been fixed in Oban Pro 1.5.1. Thanks to @sorentwo for the fix!

1 Like