Pretty_log: logfmt (and other formats) Logger formatter

Here in Ispirata we develop Astarte, an Elixir IoT/data processing platform. Astarte is made of several microservices which produce a massive amount of logs.

Recently we decided to start using logfmt as logs format, which is a human (and machine) readable structured log format, that makes queries, filtering and analysis easy while keeping log lines clear for humans.
Last but not least it avoids bikeshedding about how to format log lines, so developers are free to spend their time bikeshedding on more interesting topics.

To achieve this goal we’ve created pretty_log Elixir Logger formatter library, which takes care of formatting log (without the need of changing logger backend).

pretty_log is metadata centric, so it encourages use of Logger.metadata to keep track of all relevant information.
We also created a replacement plug for Plug.Logger (plug_logger_with_meta
) which logs requests with all relevant metadata.

This is how log lines look like when using both pretty_log and plug_logger_with_meta:

level=info ts=2019-10-28T16:57:41.087+01:00 msg="Got client request." method=GET request_path=/v1/autotestrealm/interfaces module=PlugMetadataLogger function=call/2 request_id=FdHav4gx9ruV1-QAACZi tag=got_client_req
level=info ts=2019-10-28T16:57:41.110+01:00 msg="Sent reply." status_code=200 elapsed=22ms realm=autotestrealm module=PlugMetadataLogger function=call/2 request_id=FdHav4gx9ruV1-QAACZi tag=sent_reply

Hope you’ll find this useful :slight_smile:
Let me know if you have any request/question/feedback.

8 Likes

Nice.

And for those looking for similar for Erlang’s logger there is https://github.com/ferd/flatlog – not sure if this can work with Elixir yet… I believe you can use logger as the backend for Logger now but not sure the metadata works.

2 Likes