What do we do with logging in libraries?

Good article buy I was surprised Syslog 5424 wasn’t mentioned at all. No need to come up with a format while there’s already a standard which includes both structured data and human-readable portion.

Well, TIL about RFC / syslog 5424.

I use a telemetry based logger in my libs (gen_mcp : gen_mcp/lib/gen_mcp/telemetry_logger.ex at main · lud/gen_mcp · GitHub , crown : crown/lib/crown/telemetry_logger.ex at main · lud/crown · GitHub )

I believe you should expand that part of the post, or make a follow post, because it’s the best solution (to me at least): it lets people enable the logs they want, and does not get in the way.

Hmm, I admit I don’t have a lot of experience with Syslog 5424, but I think it’s a format, right? Just like GCP or Datadog json-based formats? In that case, it’s not really relevant here, since I’m not proposing a new export format, but rather a way to write logs that will better accommodate any format. In general case, a running app can have multiple logging handlers: a default handler with default terminal formatter, an additional handler with json format and another one that sends errors to Sentry over HTTP. Reports are equipped to work well with all of them.

I touched this topic at the end. Telemetry doesn’t really solve this particular problem. In your library, you still have a handler that calls Logger.log with dynamic message and metadata, which is ok, but not perfect. If users were to customize logs, they would have to basically throw away all your efforts and do the work themselves. Telemetry just makes it easier to do.

You could totally use reports in your handler and that would make your logs even better!

I touched this topic at the end

I know and mentioned it

Telemetry loggers like mine have advantages too, it’s opt-in, filtering by prefixes is easy and straightforward, and it’s telemetry, you are dispatching an event that can be logged, but can also be used for other purposes such as converting to OTel metrics.

But I don’t disagree with your post at all, actually I think it’s very interesting.

You could totally use reports in your handler and that would make your logs even better!

Yes! A basic telemetry logger could in addition call the logger with the whole telemetry metadata and report_cb instead of converting to a string. So users can attach the default logger and filter / reformat, or they can filter at the producer level (and still reformat).

they would have to basically throw away all your efforts and do the work themselves

This is a good thing in my opinion. It’s basic code that is really easy to write. Writing a filter is more or less the same level of effort, but attaching your own telemetry handler lets you start with a blank slate.

RFC 5424 is just the output format, not the approach to firing logs, as @martosaur pointed out. This format is also quite painful to use in many places, because you need to apply for Private Enterprise Number if you want to conform to it. And SD-DATA from RFC 5424 is more for metadata rather than message itself (there is difference, substantial difference, between these two).

TBH the same applies to report/structured logs. Telemetry is log dispatching mechanism (very primitive one), just focused mostly on metrics.

This feels very much in line with me favoring using exception modules (Example) to build exception messages from data – even though I myself am not the best in following that advice.

But besides the technical question there’s also the “what is worth logging and how do I decide on a severity” portion, especially when working on a library. Because if that one’s not right then people need to start filtering out logs and I personally prefer to work additive rather than logging stuff only for it then needed to be filtered. That’s what I personally struggle more with. Maybe that’s why I tend to debug in production using traces. I know they’re not useful when you cannot reproduce an issue, but when you can they’re just so much more flexible.

Edit:

I just looked at logger — OTP 29.0.3 (kernel 11.0.3). It seems like there’s also a arity 2 version, which would allow replacing the :io formatting with a custom to string formatting. That could be used to make this a bit more approachable from elixirs end.

That is why I am not much of a fan of Elixir’s default of Logger level set to :debug. Erlang which sets it by default to :notice is IMHO better one, as it (by default) filter more messages that are irrelevant to the user. It has slight disadvantage in development, but IMHO we should make defaults safe for production, not for development.

Anyway, I digress a little.

Actually it is the exact opposite. The reason for that is that when you use report_cb/2 then it is your requirement to ensure, that for example line length is preserved. If you use report_cb/1 it is left to the formatter to keep all that requirements.

I understood that. But it means :io.format is no the only possible option. I do agree with the blog post that this is quite the arcane formatting syntax, which I’ve never really seen favored by people using elixir. There could be alternatives still handling all those constraints, but using a different syntax for formatting.