Oban add_workflow/4 with put_context/2 in a sub-workflow

You’re correct. The Context job is virtual and shouldn’t ever run.

Are you seeing this issue in testing, or when running the jobs normally? We’ll take a look into it either way, but there are semantic differences

I am seeing this issue in testing. I actually do not believe it appears when running the job normally, but I have refactored the child workflows to do something like this:

parent_workflow_id = Keyword.get(opts, :parent_workflow_id, nil)

workflow =
  case parent_workflow_id do
    id when is_binary(id) ->
      Workflow.new()

    nil ->
      Workflow.new()
      |> Workflow.put_context(%{
        source_name: to_string(source_name),
        batch_size: batch_size
      })
  end

And just make sure I have the orchestration workflow set the correct context. I can un-refactor it if you want me to check the difference between test/dev environments, but I do recall it works fine outside of test.

Are you testing with inline mode? If so, that would cause a problem. Workflows aren’t really compatible with inline mode because the jobs never hit the database. They are executed in memory in insert order, which doesn’t exercise the workflow dependencies at all.

Testing workflows should be done with the run_workflow/1 Pro testing helper, or manual draining from manual testing mode if needed.

I am testing in :manual mode. An example test setup looks something like this:

Orchestration.kickoff(source_name, pipeline: :daily)
assert %{completed: 8} = drain_jobs()

Sub-workflows have been tested and work correctly like so:

MapperWorkflow.start(source_name)
assert %{completed: 2} = drain_jobs()

I’ll give run_workflow/1 a shot real quick and see if it solves the issue

Okay I tried run_workflow/1 and also re-checked in dev mode with my workaround removed. Both cases actually act identical to testing in :manual mode with drain_jobs/1.

I.E.: ** (RuntimeError) module is not a worker: Oban.Pro.Workers.Context

When actually trying this out at runtime in dev mode, the workflow actually runs correctly, but the MapperWorkflow’s Oban.Pro.Workers.Context fails with the following meta:

%{
...
}

The MapperWorkflow then continues on to work correctly, I assume with the parent’s context (though I haven’t validated whether that is true)

Thanks for investigating and confirming it’s the same result with run_workflow. We’ll get it fixed

Update on this—we were able to reproduce the issue and there’s a fix on main ready to go out with the next patch. Thanks again for such detailed investigation work!

Great stuff thanks for looking into it!