martosaur

martosaur

Workflow dependency may not be respected in some cases if grafted workflow dynamically appends job

Hi! Our team ran into an issue where a job would not respect a dependency on a grafted workflow, if this workflow dynamically appends job to itself. Here a short reproduction script:

defmodule WaitJob do
  use Oban.Pro.Worker, queue: :not_default
  
  @impl true
  def process(%{args: %{"i" => i}}) do
    IO.inspect(i, label: "START WAIT")
    Process.sleep(30_000)
    IO.inspect(i, label: "END WAIT")
    
    :ok
  end
end

defmodule SubWorkflow do
  use Oban.Pro.Worker

  alias Oban.Pro.Workflow

  def workflow() do
    Workflow.new()
    |> Workflow.add(:sub_a, new(%{}))
  end

  @impl true
  def process(job) do
    waits = Enum.map(1..5, &WaitJob.new(%{i: &1}))

    job
    |> Workflow.all_jobs()
    |> Workflow.append()
    |> Workflow.add_many(:long_wait, waits)
    |> Oban.insert_all()

    :ok
  end
end

defmodule MainWorkflow do
  use Oban.Pro.Worker

  alias Oban.Pro.Workflow

  def workflow() do
    Workflow.new()
    |> Workflow.add_graft(:a, &graft_a/1)
    |> Workflow.add(:b, new(%{}), deps: :a)
  end
  
  def graft_a(_) do
    SubWorkflow.workflow()
    |> Workflow.apply_graft()
    |> Oban.insert_all()
  end

  @impl true
  def process(_job) do
    IO.inspect("PARENT WORKFLOW COMPLETED")

    :ok
  end
end
iex(46)> MainWorkflow.workflow() |> Oban.insert_all()
"PARENT WORKFLOW COMPLETED"
START WAIT: 3
START WAIT: 1
START WAIT: 4
START WAIT: 2
START WAIT: 5
END WAIT: 3
END WAIT: 1
END WAIT: 4
END WAIT: 2
END WAIT: 5

From my testing, this only seems to happen if appended jobs use a different queue.

Most Liked

sorentwo

sorentwo

Oban Core Team

@martosaur Strange :thinking:. We based the fix on a minimal reproduction of your reproduction, which did initially fail before the fixes. Taking another look for v1.6.11

Where Next?

Popular in Questions Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31494 112
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 40042 209
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

We're in Beta

About us Mission Statement