natanaelsirqueira

natanaelsirqueira

Oban Web/Met counting jobs twice in a scenario with separate Web and Worker nodes

Hello! I need help setting up Oban Web with separate Web and Worker nodes.

My scenario is:

  • The Worker app runs the ObanRunner instance with all our queues.
  • The Web app has a separate ObanWatcher instance that I’m passing as the :oban_name option to Oban.Web.Router.oban_dashboard/2; This instance is configured with queues: false since it’s not a worker node.
  • Both the ObanWatcher and ObanRunner instances have Oban.Met in the plugins list and auto-start is disabled globally.
  • The Worker and Web apps are released/deployed separately.

The problem is that:

  • If the Worker and Web nodes are clustered together, the job counts in the ‘States’ sidebar component are shown doubled (i.e. if there are 10k completed jobs in the database, it will show 20k in the dashboard).
  • If the Worker and Web nodes are not clustered together, the counts are accurate but the ‘Nodes’ and ‘Queues’ sidebar components won’t show (I presume because of the queues: false option).

This seems to be an issue in the Oban.Met package, as a similar scenario using the previous Oban.Web.Plugins.Stats plugin would work.

Is there anything I’m doing wrong? I’m aware that turning Worker into a Phoenix app so the dashboard can talk directly to the ObanRunner instance should work, but I would like to avoid that since each app serves a different purpose.

PS: I already tried switching to the PG notifier (with the two nodes in the same cluster) and I also confirmed it’s not related to the estimate function; I didn’t find any other possible reasons in the documentation for oban_web or oban_met.

Marked As Solved

sorentwo

sorentwo

Oban Core Team

You aren’t doing anything wrong, but you’ve hit a quirk of having two instances with different names pointed at the same table and on the same network. The solution is to set peer: false on the Web nodes to prevent them from having leadership.

Internally, Met will report counts from each leader. The other instances will record those values and sum them together. If Web and Worker used the same instance name, e.g. Oban, only one would be leader and this wouldn’t be an issue.

It’s kind of a feature that instances with different names can share metrics, but it can be confusing and Met shouldn’t be summing the values regardless.

Also Liked

natanaelsirqueira

natanaelsirqueira

Thanks for the quick and thorough response! It works perfectly now. :purple_heart:

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement