Separating app-level debug logging from Ecto debug logging

dogweather

I have some issues with logging in Elixir / Phoenix / LiveView.

In production, I want to see simple one-line logs for web requests and “important” worker processes that get kicked off.

In development, I usually want to only see my particular debug messages for the issue I’m working on.

Occasionally in development, I also want to see full database query logging.

Lol, this is what gpt-4.1 says:

You’ve clearly articulated the classic Elixir/Phoenix logging pain:

:debug is too noisy (full of Ecto queries, etc.)

:info is used by Phoenix for web requests, so your own debug/info logs get mixed in

You want a way to see your own debug logs in dev, but not get drowned out by Ecto, and you want clean, important logs in prod.

Now, I’ve found that the system by default (and with a few modifications I’ve made) gives:

:debug level — database queries (very high volume & noisy output)
:info level — web requests

So, when I use :debug for my app debugging, those messages get lost in the db query log noise. This is the problem I’m looking to solve.

In search of a solution, I found this documentation:

Levels

The supported levels, ordered by importance, are:

:emergency - when system is unusable, panics
:alert - for alerts, actions that must be taken immediately, ex. corrupted database
:critical - for critical conditions
:error - for errors
:warning - for warnings
:notice - for normal, but significant, messages
:info - for information of any kind
:debug - for debug-related messages

For example, :info takes precedence over :debug. If your log level is set to :info, then all :info, :notice and above will be passed to handlers. If your log level is set to :alert, only :alert and :emergency will be printed.

So, I’m considering using :info for my app-level debug logging, and :notice for production info-type messages — web requests and important events:

Change default :info to use :notice. E.g., web requests.
I never use :debug, leaving it the province of Ecto and other 3rd party libraries.
I use :info only for my app code debugging.
In production, I set the level threshold to :notice.

Are there any other alternatives I should consider?

3 comments

/phoenix #logging

0 534 3

2025-07-29 00:54:15 UTC

Showing Posts 1 to 3

Show Best Posts
Show All (oldest first)
Show All (newest first)

garrison

The problem is not on the logging side, it’s on the filtering side (output, not input). There are many techniques to filter logs, ranging from a text-based approach (grep) to complex filtering on structured or semi-structured logs. There are platforms that support SQL, even.

I don’t know if there are any tools for this for development, though. Funny, recently I was watching an old stream from one of the Tigerbeetle devs debugging a simulation failure. Since the simulator in full debug mode generates several gigs of logs he had written a tool to filter them live in his editor. It looked to be pretty simple, just some accept/reject filters that recompute a view of the log file. Maybe we need something like that!

But yeah, adding more log levels is never going to solve this IMO.

Post #1

martosaur

Sounds reasonable. But tbh any more or less used production environment is likely to generate so many logs that plain text view of them is hardly useful. Most log aggregators can filter out logs pretty efficiently, so that would be the first tool I would reach for.

Post #2

axelson

Scenic Core Team

It’s possible to use code to change the log level on a per-module basis. So you could use that to remove logs from other portions of the code base especially if you have a consistent module prefix to use.

Post #3

— All posts loaded —