Mukulch15
Parallel job processing in Oban
I have created a queue in Oban with a concurrency of 12. The worker’s job is to take the parameters, make an http call and insert the response into database. However when I inserted jobs using Oban.insert_all, I see that the jobs are getting processed serially and not in parallel. Am I missing any config ? Or is it something provided in Oban pro.
Marked As Solved
sorentwo
Ah. This is because you’re running with inline test mode. That should only be enabled in test.exs, not for your base config.
The way you have it configured you’re never persisting anything into the database, it’s executing them one at a time in memory.
Also Liked
Mukulch15
outlog
is the http library running a queue of 1?
else show some pseudo code, and/or debug by removing http call and do something like this in the worker:
IO.inspect("job started")
:timer.sleep(5_000)
IO.inspect("job done")
and see if they are parallel..
benwilson512
If you have 12 workers, they should all try to process a job, which means you should see 12 jobs in progress at once (parallel).
Can you tell us how you are sure they’re happening serially?
Popular in Questions
Other popular topics
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"