Thoughts/suggestions for a complex Oban Pro Worklow

Questions / Help
, ,
1

I’m looking for thoughts on how to best handle an expansion I need to work into a current Oban.Pro.Workers.Workflow we have in place to handle order payment verfication and publishing logic:

This OrderCreated workflow has three steps:

  1. job to poll for a record’s status being changed due to separate async webhook job handling that payment event

  2. job for recording some data and updating the order status accordingly

  3. job to publish the order on our internal bus

I’m working on incorporating checking some risk analysis data in certain cases. My first blush approach is adding the “certain cases” config and checking that in job #2. If we think the risk data warrants manual review we stamp that status on the order and kill the workflow. This prevents publishing, which is definitely what we want until the manual review is done.

My thought going forward is to queue a new job that polls for a new notification (another webhook) indicating that the manual check is done. That could even be another Workflow… but it seems like it would be very similar to this current workflow. I wonder if I’m missing something about Oban.Pro.Workfows though? Some way to send it back to the polling step?

I suppose I could use the dynamic appending feature to inject an alternate set of steps into the workflow if we need to do the risk analysis. The original step 2 would stay as-is for the non-analysis case but the risk flow would do a separate step 2 that checks if we need manual review and then start a polling job (ie: with a backoff) until we get the approval wehhook (or it times out).

One question I have that I don’t think the docs cover is can I add a worker dynamically to a Workflow from within a job in the workflow? If so, that would be an alternative to checking the configuration at the time the workflow is queued. Not sure if it would be better though.

I asked this in Slack and someone suggested I look at Oban.Pro.Workers.Chain instead of Workflow and on a quick glance it does look like, in the case of manual risk review, I could snooze the workflow for X amount of time. I’ll take a closer look at that but I still wanted to post this here on the forum, both to get more :eyes: and simply because Parker asked me to. :slight_smile:

Thanks!

2 Likes
2

As of Pro v1.4 workflows no longer poll. A job with dependencies is scheduled in an “on-hold” state, which is then resolved when all the dependencies are completed. Of course, you can use a snooze loop from your own job and delay downstream jobs in the workflow:

def process(job) do
  if thing_is_finished?(job.args) do
    :ok
  else
    {:snooze, 5}
  end
end

You absolutely can with append/2. It’s designed to be started with a job within the workflow:

def process(job) do
  # do something in the process function

  job
  |> Workflow.append()
  |> Workflow.add(:new_step, WorkerD.new(%{}), deps: [:old_step])
  |> Oban.insert_all()

  :ok
end

Chains are a tool to force an unbounded stream of jobs to run in sequential order, whereas a Workflow is meant for a finite set of jobs. Your use case seems like a good fit for a Workflow.

That’s a good enough reason for me :wink: (seriously, thanks; it’s appreciated)

3

Thanks! After further reflection and digging into how the results of the analysis are processed, I’m now thinking I may not need to append a job after all. It seems like this initial workflow can just finish and there are separate jobs/workflows that handle the results of the analysis. But good to know the details you present here. Cheers.

1 Like