martosaur

martosaur

What do we do with logging in libraries?

Seriously, what do we do with logs in Elixir? There is seemingly no way to make them good both in dev and prod. This is especially noticeable in libraries, which often choose to simply not log.

In my new post I propose a solution and explain why telemetry isn’t quite it.

TL;DR: structured logging aka reports logging is a great way to make your logs programmatically accessible and thus customizable while still providing a preferred, terminal-friendly text representation

First Post!

krasenyp

krasenyp

Good article buy I was surprised Syslog 5424 wasn’t mentioned at all. No need to come up with a format while there’s already a standard which includes both structured data and human-readable portion.

Most Liked

videsnelson

videsnelson

I really can’t miss this opportunity to scream that Elixir logs have always been one of my biggest and most frequent rants. For all that is good about this language, logs are unforgivably horrendous :sweat_smile:

I’d love if unstructured logs did not exist, ever. There’s not a single good usecase where structured logs plus a report_cb doesn’t win. I always create structured logs with a report_cb that formatters can optionally call if they want to create a user-friendly string from the structure, noting, “user-friendly” is forever a subjective opinion and should therefore be strictly optional and never overriding the actual structure, in case someone else wants to format things differently. And even then, the “user-friendly” string should not replace the structured payload. But then I’ve seen many libraries that, as soon as they see a report_cb field, replace the entire structure with the value of that report_cb, so I can’t even safely add my own callbacks either.

I’d also be more than happy if Logger just did not exist at all, for once erlang’s :logger is better at about everything. For example, how the Logger is configured always bites me, it’s all :logger under the hood but all the configuration is slightly different so it always takes me several rounds to configure the right thing (one example, handlers other than the default are configured under the application and with a few keys being named differently).

If there’s only one thing I wish to put out there and for every one to take, ranting aside, is to use structured logging, not optionally, not sometimes, but as the only valid way. I’d even make it enforced by the library and fail compilation otherwise :pleading_face:

PS: @PragTob has heard my rants so many times :smiling_face_with_tear:

martosaur

martosaur

Hmm, I admit I don’t have a lot of experience with Syslog 5424, but I think it’s a format, right? Just like GCP or Datadog json-based formats? In that case, it’s not really relevant here, since I’m not proposing a new export format, but rather a way to write logs that will better accommodate any format. In general case, a running app can have multiple logging handlers: a default handler with default terminal formatter, an additional handler with json format and another one that sends errors to Sentry over HTTP. Reports are equipped to work well with all of them.

martosaur

martosaur

I touched this topic at the end. Telemetry doesn’t really solve this particular problem. In your library, you still have a handler that calls Logger.log with dynamic message and metadata, which is ok, but not perfect. If users were to customize logs, they would have to basically throw away all your efforts and do the work themselves. Telemetry just makes it easier to do.

You could totally use reports in your handler and that would make your logs even better!

Last Post!

LostKobrakai

LostKobrakai

I’m not sure how useful this is by itself tbh. I see this purely as an implemenation detail of how :logger handles turning the information into text. I would never reuse those callbacks for anything else.

Where Next?

Popular in Blog Posts Top

brainlid
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
New
RudManusachi
Hi there! Recently I was playing around with extracting and updating data in the DB and for fun challenged myself to try to implement a ...
New
brainlid
OTP 26 was released and the Elixir 1.14.4 builds have been updated! Erlang OTP 26 changed how map keys are sorted, or not sorted actually...
New
PragTob
I ran into an interesting problem recently where simple concurrency on the BEAM via Task.async made my application a lot slower and a lot...
New
rlopzc
Use the new log handlers to plug Slack or any other provider into your logging system. https://rlopzc.com/posts/integrate-slack-into-the...
New
lawik
One of the Erlang ecosystem’s spiciest nerd snipes are hot code updates. Because it can do it. In ways that almost no other runtime can.
New
lawik
Building on other people’s work I bashed things together and suddenly I can know when someone is speaking using Elixir and Membrane.
New

Other popular topics Top

New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 44778 311
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement