Question about `after_process/3` callback in a `Oban.Pro.Workers.Workflow`

[adammokan]

I’ve used the after_process/3 callback a number of times when implementing a Oban.Pro.Worker and it tends to work great for handling any cleanup or updating something in the database based on the job status.

I have a situation today where I’m implementing a Oban.Pro.Workers.Workflow with up to 3 dynamic child stages appended during execution and I realized I want to have a final callback that allows me to update a supporting record in the DB when the entire workflow completes. Reached for the after_process callback and instinctively added that to my first workflow stage, but (logically in hindsight) it fires right when that first stage completes rather than the entire workflow itself.

My primary question here is if I am missing something on how to handle this via a callback when using workflows? Or is the answer to append that final step I hoped to handle in the callback as a child of the workflow as its own job or is it recommended to attach that callback to the final stage of my workflow?

I think what makes this a bit confusing is that we do have a workflow-centric callback in the form of after_cancelled/4. So it seems like we should have some way to do the same for workflows that were not cancelled?

Appreciate any insight on this!

Update: I did just realize after_cancelled/4 is brand new as of last week. So maybe a similar approach for success cases is in the works

[sorentwo]

@adammokan

My primary question here is if I am missing something on how to handle this via a callback when using workflows? Or is the answer to append that final step I hoped to handle in the callback as a child of the workflow as its own job or is it recommended to attach that callback to the final stage of my workflow?

You’re not missing anything—appending a final job as the last step or adding an after_process/3 callback to the final job in the workflow is the way to solve this. I’d suggest a final job over a callback because you’ll have all the niceties of a job (i.e. retries), and it’s more obvious if you look at the workflow as a whole.

@adammokan

I think what makes this a bit confusing is that we do have a workflow-centric callback in the form of after_cancelled/4. So it seems like we should have some way to do the same for workflows that were not cancelled?

That callback is brand new (as noted in your update), and it was added to handle a situation that was impossible before. Prior to after_cancelled/2, there wasn’t any way to react to downstream workflow jobs getting cancelled because they weren’t actually running.

@adammokan

So maybe a similar approach for success cases is in the works

This use-case comes up frequently and a first-class approach for “batch callback” functionality in workflows is in the works