How to silence Oban during an iex session?

Special Sections Oban
,
1

@sorentwo I’m running into an issue where I start an iex session (iex -S mix) and then my terminal keeps showing log messages from Oban:

iex(9)> [info] {"event":"notifier:switch","message":"notifier can't receive messages from any nodes, functionality degraded","source":"oban","status":"isolated"}
[info] {"event":"notifier:switch","message":"notifier only receiving messages from its own node, functionality may be degraded","source":"oban","status":"solitary"}
[info] {"event":"notifier:switch","message":"notifier can't receive messages from any nodes, functionality degraded","source":"oban","status":"isolated"}
[info] {"event":"notifier:switch","message":"notifier only receiving messages from its own node, functionality may be degraded","source":"oban","status":"solitary"}
[info] {"event":"notifier:switch","message":"notifier can't receive messages from any nodes, functionality degraded","source":"oban","status":"isolated"}
[info] {"event":"notifier:switch","message":"notifier only receiving messages from its own node, functionality may be degraded","source":"oban","status":"solitary"}

When I run Application.get_env(:app, Oban), I get:

[
  repo: App.PostgresRepo,
  engine: Oban.Engines.Basic,
  queues: [],
  plugins: [],
  log: false
]

(I use the Pro Engine but opted for Basic as it seemed like these messages might be related).

Any idea how to quiet the logs so I can use iex without being constantly interrupted by Oban?
Scott

2

I suspected these logs are from telemetry events it looks like the case here: oban/lib/oban/telemetry.ex at febfc63d8e7afab62ab9a0da9c782d6374f4a752 · sorentwo/oban · GitHub

You could probably put something in your .iex.exs that disables the logger entirely:

# .iex.exs
:ok = Oban.Telemetry.detach_default_logger()

That ought to work, but you could experiment with different log levels and configure your logger to ignore logs lower than info, while setting Oban’s logger to debug. Alternatively, find where you’re calling attach_default_logger and simply code around it if you’re in a certain MIX_ENV.

4 Likes
3

Those particular messages are fixed in Oban v2.17.5. You should only see one message, not have it repeat every 15 seconds.

The top level log option is for Ecto logging (the name isn’t very clear). The logs you’re seeing are from Oban.Telemetry.attach_default_logger. The proper way to silence that in iex is with Oban.Telemetry.detach_default_logger(), as noted by @beepbeepbopbop

4 Likes
4

Thank you @beepbeepbopbop and @sorentwo !

The upgrade to 2.17.5 fixed the immediate problem. Thanks!

5

Unless I’m missing something, I just installed Oban 2.17.8 and I’m still seeing this message repeated every 15 seconds locally (running mix phx.server)

Detaching the default logger in .iex.exs doesn’t seem to help either. What am I doing wrong?

6

Have you tried recompiling and restarting? That issue is definitely fixed in versions since v2.17.5

7

It probably is user error. It didn’t seem to be periodic, but these messages were printed definitely more than once:

[info] [message: "notifier only receiving messages from its own node, functionality may be degraded", status: :solitary, source: "oban", event: "notifier:switch"]
[info] [message: "notifier can't receive messages from any nodes, functionality degraded", status: :isolated, source: "oban", event: "notifier:switch"]

Maybe they were triggered by code recompilation, I’m not sure.

In any case, I removed the default logger and wrote my own Oban handler since I have to deal with forwarding exceptions to Sentry anyway.

8

Glad you solved it by writing your own logger. Note that those messages are important in production, as they indicate whether core functionality like cancelling/starting/stopping/pausing/resuming queues across nodes will actually function.

Side note, the most recent versions of Sentry have integrated Oban support: Oban Integration — Sentry v10.3.0

9

Yes but I am afraid that my single-node app would fill its log with that warning. I wish it was possible to disable it altogether on single node environments, such as local development.

Thanks for the Sentry note, I will look into that.

10

That warning, and the event that triggers it, are only emitted every 15s if the status changes: oban/lib/oban/sonar.ex at main · sorentwo/oban · GitHub. This is the first report of extraneous messages since v2.17.5 was released, and I’d worry something is wrong with your environment.

Regardless, it definitely should only happen once in a single node production environment.

11

Here’s another approach I use to access iex and avoid noise from logs / recompilation messages / etc, with a couple simple cli aliases so that logs appear in the first one but not the extra one:

dev → iex --sname localenv -S mix phx.server

dev-extra → iex --sname extra --remsh localenv

1 Like
12

I’m getting this message periodically (even without rebuliding) in a local development in a fresh Phoenix install. Mix says I’m running Oban 2.17.10. Is there any way to suppress it in a development environment?

[info] {"message":"notifier only receiving messages from its own node, functionality may be degraded","status":"solitary","source":"oban","event":"notifier:switch"}

Rebuilding...

[info] {"message":"job staging switched to local mode. local mode polls for jobs for every queue; restore global mode with a functional notifier","mode":"local","source":"oban","event":"stager:switch"}
[info] {"message":"job staging switched back to global mode","mode":"global","source":"oban","event":"stager:switch"}
[info] {"message":"notifier only receiving messages from its own node, functionality may be degraded","status":"solitary","source":"oban","event":"notifier:switch"}
[info] {"message":"notifier can't receive messages from any nodes, functionality degraded","status":"isolated","source":"oban","event":"notifier:switch"}
[info] {"message":"notifier only receiving messages from its own node, functionality may be degraded","status":"solitary","source":"oban","event":"notifier:switch"}

EDIT: I’m just going to not use Oban.Telemetry.attach_default_logger when Mix.env() == :dev for now; getting notifier:switch events is adding clutter that makes it harder to see more useful logs. I wish there were an option to suppress notifier:switch messages only, but it looks like there isn’t one.

13

In my case, I updated to oban 2.17.10, running single-node app as well, but I still see these messages every 15-20 seconds.

13:24:34.242 [info] {"message":"notifier can't receive messages from any nodes, functionality degraded","status":"isolated","source":"oban","event":"notifier:switch"}
13:24:34.242 [info] {"message":"notifier only receiving messages from its own node, functionality may be degraded","status":"solitary","source":"oban","event":"notifier:switch"}

I cleared the dependencies and recompiled them. However, I wonder, If this issue was solved then is there something else to be updated configuration-wise/database?

14

Which environment is this in, and which notifier are you using?

Those two messages have the exact same timestamp. Is it always like that?

1 Like
15

It turns out there are two Oban processes in the umbrella project, thank you for spotting that out. I will read again the documentation.
Cheers