We’ve been using ObanWeb and ObanPro successfully for years, but as of late, we are consistently encountering all zeros in the dashboard job counts, despite many jobs obviously executing. We see a count for total jobs executing in the Nodes
section, but everything else is 0. After a deploy, the counts work briefly, but will quickly revert to all 0s. We haven’t made any changes to our Oban config lately. We’re not seeing any errors making it to our error reporting service - it is maybe possible that something is being logged but not reported. We’re currently on oban_web
2.10.2. Thanks for any help!
Counts and queue state information all comes from distributed metrics. There are a couple of components that must be operational for those metrics to flow, and make it to the dashboard. As the dashboard was working, it seems like either a leadership or pubsub issue:
- Leadership—only the cluster leader, which you can check with
Oban.Peer.get_leader()
, will perform database counts. - PubSub—all nodes running jobs must be connected to the same cluster, and the Web frontend must be connected as well.
Which notifier are you using? If you didn’t configure anything, it’s the default Postgres notifier, which may have connectivity problems. The Web installation guide recommends the PG notifier for performance and accuracy.
Thanks for the reply. We’re using the default notifier, so it seems like an easy first step will be to try out the recommended Oban notifier and see if that solves the issue. Will keep you posted!
Did you get this issue resolved? I see the same issue on my app
I did not unfortunately. I tried swapping in the Oban notifier, and that didn’t do the trick. The Oban.Peer.get_leader
func that sorentwo references in his reply isn’t on the version of oban we’re currently using, so my next plan is to upgrade oban to the latest and check on the leader. I’ll report out once I find the time to do the upgrade.
Upgrading to oban_web
2.11.1
fixed the issue!