Parallel job processing in Oban

is the http library running a queue of 1?

else show some pseudo code, and/or debug by removing http call and do something like this in the worker:

IO.inspect("job started")
:timer.sleep(5_000)
IO.inspect("job done")

and see if they are parallel..

If you have 12 workers, they should all try to process a job, which means you should see 12 jobs in progress at once (parallel).

Can you tell us how you are sure they’re happening serially?

@outlog @benwilson512 So I tried to simulate using the above snippet as well.

IO.inspect("job started")
:timer.sleep(5_000)
IO.inspect("job done")

The output is serial. After every 5 seconds, Job Done gets printed. If it were parallel, I should have seen at least 12 get printed together.

My config:

config :App, Oban,
  repo:App.Repo,
  plugins: [Oban.Plugins.Pruner],
  queues: [test: 12],
  testing: :inline

Code:

def perform(params) do
    IO.inspect("job started")
    :timer.sleep(5_000)
    IO.inspect("job done")
end

Ah. This is because you’re running with inline test mode. That should only be enabled in test.exs, not for your base config.

The way you have it configured you’re never persisting anything into the database, it’s executing them one at a time in memory.

Thank you so much. It’s working now. It’s been more than a day.