martosaur

martosaur

Hi! Our team ran into an issue where a job would not respect a dependency on a grafted workflow, if this workflow dynamically appends job to itself. Here a short reproduction script:

defmodule WaitJob do
  use Oban.Pro.Worker, queue: :not_default
  
  @impl true
  def process(%{args: %{"i" => i}}) do
    IO.inspect(i, label: "START WAIT")
    Process.sleep(30_000)
    IO.inspect(i, label: "END WAIT")
    
    :ok
  end
end

defmodule SubWorkflow do
  use Oban.Pro.Worker

  alias Oban.Pro.Workflow

  def workflow() do
    Workflow.new()
    |> Workflow.add(:sub_a, new(%{}))
  end

  @impl true
  def process(job) do
    waits = Enum.map(1..5, &WaitJob.new(%{i: &1}))

    job
    |> Workflow.all_jobs()
    |> Workflow.append()
    |> Workflow.add_many(:long_wait, waits)
    |> Oban.insert_all()

    :ok
  end
end

defmodule MainWorkflow do
  use Oban.Pro.Worker

  alias Oban.Pro.Workflow

  def workflow() do
    Workflow.new()
    |> Workflow.add_graft(:a, &graft_a/1)
    |> Workflow.add(:b, new(%{}), deps: :a)
  end
  
  def graft_a(_) do
    SubWorkflow.workflow()
    |> Workflow.apply_graft()
    |> Oban.insert_all()
  end

  @impl true
  def process(_job) do
    IO.inspect("PARENT WORKFLOW COMPLETED")

    :ok
  end
end
iex(46)> MainWorkflow.workflow() |> Oban.insert_all()
"PARENT WORKFLOW COMPLETED"
START WAIT: 3
START WAIT: 1
START WAIT: 4
START WAIT: 2
START WAIT: 5
END WAIT: 3
END WAIT: 1
END WAIT: 4
END WAIT: 2
END WAIT: 5

From my testing, this only seems to happen if appended jobs use a different queue.

Showing Posts 1 to 2

martosaur

martosaur OP

Looking at 1.6.10 changelog, this one should be fixed. But after removing the workaround in our app I don’t see any difference in behaviour :thinking: I tried the reproduction script from the initial post and it behaves the same way.

@sorentwo could you please double check if I’m missing something here?

sorentwo

sorentwo

Oban Core Team

@martosaur Strange :thinking:. We based the fix on a minimal reproduction of your reproduction, which did initially fail before the fixes. Taking another look for v1.6.11

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
asweet-confluent
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
ryanwinchester
apply_graft/2 doesn’t rewrite an add_many sub-workflow’s deps on an add step. Grafted jobs cancel with “upstream job was deleted” Version...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews