pwightman
Understanding queries made by Oban Pro
Hi,
We have a Phoenix/LiveView app, with web servers and separate jobs servers that run Oban Pro. There are 4 jobs servers running.
When we look at our database metrics, what appear to be internal Oban queries seem to dominate the load on our database so we’re trying to understand this better.
At all times, but particularly during load spikes, our load visualizer shows this during a 5 minute period:
As you can see, one query at the top is called 33,384 times in 5 minutes, and it’s this query:
SELECT
o0."state",
o0."scheduled_at"
FROM
"public"."oban_jobs" AS o0
WHERE
(o0."meta" @> $1)
AND (o0."state" != $3)
AND (o0."id" < $2)
ORDER BY
o0."id" DESC
LIMIT $4;
I’ve searched our code pretty exhaustively and I’m unable to find this query, so it seems like it might be internal to Oban.
We’re not sure where to find more information on this (my Google-fu fails me, and might be part of Oban Pro?) to understand why it’s being called this much and how to diagnose/reduce the number of calls, if possible.
Anyone have any ideas?
Trending in Questions
Other Trending Topics
Latest Phoenix Threads
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Marked As Solved
sorentwo
That looks like the query a
Chainworker would use in versions of Pro prior to v1.5. That query is called before each chain job runs, and that can really add up with a busy queue.The good news is that query is gone and the approach is completely different in v1.5. There’s a description of the change and a small demo in one of the launch week announcements.
Also Liked
pwightman
This looks like this could really fix our problem, this is wonderful!
We have a large production app that runs LOTS and lots of jobs, I see that 1.5 is a release candidate, but do you feel like it’s pretty stable at this point? Fully ready for production?
Obviously we’ll do our own testing, but just looking for a gut check on how cautious we should be in this upgrade.
Thank you so much for this response!
Last Post!
sorentwo
Responded in our email chain. I’ll report back here when we understand what’s going on with pruning discarded jobs for your system.