pwightman

pwightman

Understanding queries made by Oban Pro

Hi,

We have a Phoenix/LiveView app, with web servers and separate jobs servers that run Oban Pro. There are 4 jobs servers running.

When we look at our database metrics, what appear to be internal Oban queries seem to dominate the load on our database so we’re trying to understand this better.

At all times, but particularly during load spikes, our load visualizer shows this during a 5 minute period:

As you can see, one query at the top is called 33,384 times in 5 minutes, and it’s this query:

SELECT 
    o0."state", 
    o0."scheduled_at" 
FROM 
    "public"."oban_jobs" AS o0 
WHERE 
    (o0."meta" @> $1) 
    AND (o0."state" != $3) 
    AND (o0."id" < $2) 
ORDER BY 
    o0."id" DESC 
LIMIT $4;

I’ve searched our code pretty exhaustively and I’m unable to find this query, so it seems like it might be internal to Oban.

We’re not sure where to find more information on this (my Google-fu fails me, and might be part of Oban Pro?) to understand why it’s being called this much and how to diagnose/reduce the number of calls, if possible.

Anyone have any ideas?

Marked As Solved

sorentwo

sorentwo

Oban Core Team

That looks like the query a Chain worker would use in versions of Pro prior to v1.5. That query is called before each chain job runs, and that can really add up with a busy queue.

The good news is that query is gone and the approach is completely different in v1.5. There’s a description of the change and a small demo in one of the launch week announcements.

Also Liked

pwightman

pwightman

This looks like this could really fix our problem, this is wonderful!

We have a large production app that runs LOTS and lots of jobs, I see that 1.5 is a release candidate, but do you feel like it’s pretty stable at this point? Fully ready for production?

Obviously we’ll do our own testing, but just looking for a gut check on how cautious we should be in this upgrade.

Thank you so much for this response!

Where Next?

Popular in Questions Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

We're in Beta

About us Mission Statement