martosaur

martosaur

What do we do with logging in libraries?

Seriously, what do we do with logs in Elixir? There is seemingly no way to make them good both in dev and prod. This is especially noticeable in libraries, which often choose to simply not log.

In my new post I propose a solution and explain why telemetry isn’t quite it.

TL;DR: structured logging aka reports logging is a great way to make your logs programmatically accessible and thus customizable while still providing a preferred, terminal-friendly text representation

First Post!

krasenyp

krasenyp

Good article buy I was surprised Syslog 5424 wasn’t mentioned at all. No need to come up with a format while there’s already a standard which includes both structured data and human-readable portion.

Most Liked

videsnelson

videsnelson

I really can’t miss this opportunity to scream that Elixir logs have always been one of my biggest and most frequent rants. For all that is good about this language, logs are unforgivably horrendous :sweat_smile:

I’d love if unstructured logs did not exist, ever. There’s not a single good usecase where structured logs plus a report_cb doesn’t win. I always create structured logs with a report_cb that formatters can optionally call if they want to create a user-friendly string from the structure, noting, “user-friendly” is forever a subjective opinion and should therefore be strictly optional and never overriding the actual structure, in case someone else wants to format things differently. And even then, the “user-friendly” string should not replace the structured payload. But then I’ve seen many libraries that, as soon as they see a report_cb field, replace the entire structure with the value of that report_cb, so I can’t even safely add my own callbacks either.

I’d also be more than happy if Logger just did not exist at all, for once erlang’s :logger is better at about everything. For example, how the Logger is configured always bites me, it’s all :logger under the hood but all the configuration is slightly different so it always takes me several rounds to configure the right thing (one example, handlers other than the default are configured under the application and with a few keys being named differently).

If there’s only one thing I wish to put out there and for every one to take, ranting aside, is to use structured logging, not optionally, not sometimes, but as the only valid way. I’d even make it enforced by the library and fail compilation otherwise :pleading_face:

PS: @PragTob has heard my rants so many times :smiling_face_with_tear:

martosaur

martosaur

Hmm, I admit I don’t have a lot of experience with Syslog 5424, but I think it’s a format, right? Just like GCP or Datadog json-based formats? In that case, it’s not really relevant here, since I’m not proposing a new export format, but rather a way to write logs that will better accommodate any format. In general case, a running app can have multiple logging handlers: a default handler with default terminal formatter, an additional handler with json format and another one that sends errors to Sentry over HTTP. Reports are equipped to work well with all of them.

martosaur

martosaur

I touched this topic at the end. Telemetry doesn’t really solve this particular problem. In your library, you still have a handler that calls Logger.log with dynamic message and metadata, which is ok, but not perfect. If users were to customize logs, they would have to basically throw away all your efforts and do the work themselves. Telemetry just makes it easier to do.

You could totally use reports in your handler and that would make your logs even better!

Last Post!

LostKobrakai

LostKobrakai

I’m not sure how useful this is by itself tbh. I see this purely as an implemenation detail of how :logger handles turning the information into text. I would never reuse those callbacks for anything else.

Where Next?

Popular in Blog Posts Top

wmnnd
Hey there, I’ve started a little blog series about building and deploying Elixir applications. Now I would like to share with you the fi...
New
ErlangSolutions
An infographic that compares Erlang, Elixir, and Go’s strengths in the respect to the programming languages’ concurrency, reliability, sc...
New
rrrene
In this series, we take a look at the different ways to organize, structure and execute a good “flow” in our Elixir programs. The latest...
New
brainlid
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
New
danschultzer
How to set up the Content Security Policy header with Phoenix LiveView and support inline styles and scripts with CSP nonces.
New
rlopzc
Learn how to leverage Erlang + Elixir’s GenServer to build a robust FTP Client. https://rlopzc.com/posts/elixir-ftp-client-genserver-par...
New
New

Other popular topics Top

vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 55125 245
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 44778 311
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement