natanaelsirqueira

natanaelsirqueira

Oban Web/Met counting jobs twice in a scenario with separate Web and Worker nodes

Hello! I need help setting up Oban Web with separate Web and Worker nodes.

My scenario is:

  • The Worker app runs the ObanRunner instance with all our queues.
  • The Web app has a separate ObanWatcher instance that I’m passing as the :oban_name option to Oban.Web.Router.oban_dashboard/2; This instance is configured with queues: false since it’s not a worker node.
  • Both the ObanWatcher and ObanRunner instances have Oban.Met in the plugins list and auto-start is disabled globally.
  • The Worker and Web apps are released/deployed separately.

The problem is that:

  • If the Worker and Web nodes are clustered together, the job counts in the ‘States’ sidebar component are shown doubled (i.e. if there are 10k completed jobs in the database, it will show 20k in the dashboard).
  • If the Worker and Web nodes are not clustered together, the counts are accurate but the ‘Nodes’ and ‘Queues’ sidebar components won’t show (I presume because of the queues: false option).

This seems to be an issue in the Oban.Met package, as a similar scenario using the previous Oban.Web.Plugins.Stats plugin would work.

Is there anything I’m doing wrong? I’m aware that turning Worker into a Phoenix app so the dashboard can talk directly to the ObanRunner instance should work, but I would like to avoid that since each app serves a different purpose.

PS: I already tried switching to the PG notifier (with the two nodes in the same cluster) and I also confirmed it’s not related to the estimate function; I didn’t find any other possible reasons in the documentation for oban_web or oban_met.

Marked As Solved

sorentwo

sorentwo

Oban Core Team

You aren’t doing anything wrong, but you’ve hit a quirk of having two instances with different names pointed at the same table and on the same network. The solution is to set peer: false on the Web nodes to prevent them from having leadership.

Internally, Met will report counts from each leader. The other instances will record those values and sum them together. If Web and Worker used the same instance name, e.g. Oban, only one would be leader and this wouldn’t be an issue.

It’s kind of a feature that instances with different names can share metrics, but it can be confusing and Met shouldn’t be summing the values regardless.

Also Liked

natanaelsirqueira

natanaelsirqueira

Thanks for the quick and thorough response! It works perfectly now. :purple_heart:

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30840 112
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47849 226
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

We're in Beta

About us Mission Statement