Godfrey

Godfrey

How to approach OBAN Pro testing workflow with 3 workers?

Good Day ,

I have a specific workflow with ,for example (Workers A,B and C) . So when worker A executes successfully, it records results which have to be used by worker B and C.
So now I want to do Unit testing on Worker B and C , how would you advise in going about that ? Because now when I run maybe a perform_job on my unit test , where will it get those recorded result since an Oban Unit test shouldn’t be hitting the DB , How do I make it pass ???

First 8 of 8 Posts Switch mode

sorentwo

sorentwo

Oban Core Team

If a job is relying on the output of another, then an integration test may be appropriate. You can test all of the jobs together with run_workflow/2:

assert %{completed: 3} =
         Workflow.new()
         |> Workflow.add(:a, MyFlow.new(%{}))
         |> Workflow.add(:b, MyFlow.new(%{}), deps: [:a])
         |> Workflow.add(:c, MyFlow.new(%{}), deps: [:b])
         |> run_workflow()

If the logic in those jobs is complex or the test requires too much setup, you can inject data through the args. The trick is using args_schema with the :term type, so you can stub any value:

defmodule MyApp.MyFlow do
  use Oban.Pro.Workers.Workflow

  args_schema do
    field :id, :id
    field :result, :term
  end

  @impl true
  def process(job) do
    {:ok, result} = fetch_lazy(job)
    ...
  end

  defp fetch_lazy(%{args: %{result: nil}} = job) do
    job
    |> Workflow.all_jobs(only_deps: true)
    |> List.first()
    |> fetch_recorded()
  end

  defp fetch_lazy(%{args: %{result: result}}), do: {:ok, result}
end

Then use perform_job/3 as you normally would in your test:

assert {:ok, _} = perform_job(MyFlow, %{id: 123, result: {:some, :thing}})
mtunski

mtunski

I don’t quite understand what you mean by saying “Oban Unit test shouldn’t be hitting the DB” - what’s wrong with that?

Anyway, in Surfer, we ended up doing the following:

  1. We insert a completed dependency job (A) into the database, with all the meaningful meta fields pre-filled (e.g. workflow_id, name, recorded & return) where the return is manually set to Oban.Pro.Utils.encode64(whatever).
  2. We insert the tested job (B) with the same meta.workflow_id and meta.deps = [job_a_name].
  3. We run drain_jobs. I guess perform_job would also do the trick here. The job B will then have access to the recorded results from job A when it executes.

After wrapping it in some helpers, you can end up with something like this:

%{foo: :bar}
|> MyWorkerB.new()
|> run_with_deps(
  job_a: %{return: :baz}
)
HataluliRandima

HataluliRandima


Hi , I saw you said you dont understand what is meant by “Oban Unit test shouldn’t be hitting the DB”, I saw on the hex docs . See attached screenshot.

sorentwo

sorentwo

Oban Core Team

The discrepancy seems to be around the term “unit”. Pure tests that don’t touch the db are a fine ideal, but the sandbox makes db interaction so quick and isolated that it’s not always worth it.

I’d argue that a workflow test, especially one that relies on fetching results of another job, should be tested with the database involved.

dimitarvp

dimitarvp

Thanks for confirming it, I always use manual testing mode and almost never the inline one.

Godfrey

Godfrey OP

Hi @sorentwo , thank you for the response . So In what situation will you use the oban unit test? or recommend one .

mtunski

mtunski

I always thought of that this way: these unit-test Oban helpers just call Worker.process/perform, without inserting the job to the database to execute it, as Oban would “normally” do.

But regardless of this behaviour, if the worker code (implementation in process/perform) itself has to reach the database - and it has if it tries reading the results of a job it depends on - it’s something different and 100% legit.

sorentwo

sorentwo

Oban Core Team

In this situation I would integration test the workflow (as described above).

— All posts loaded —

Where Next?

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
silverdr
Using Phoenix.LiveView.TagEngine as an EEx.Engine is deprecated! To compile HEEx, use Phoenix.LiveView.TagEngine.compile/2 instead. Sta...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
michallepicki
I am using Oban and occasionally, shortly after a deployment, a handful of jobs can fail because of dependency on other parts of the syst...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New

We're in Beta

About us Mission Statement