This is my mental model of current state of the loggers:

1. Logger handlers is the main interface for logging
2. Logger handlers are recommended to support something called “overload protection”
3. The default :logger_std_h handler does have an overload protection
4. If you want to write a custom handler, you’ll have to implement overload protection, like for example, Sentry handler does.
5. LoggerBackends is a logger handler which allows you to write logger backends which is a concept unique to this package. Custom backends won’t need overload protection as they are not logger handlers.

So my rule of thumb is as follow: logger handlers that implement overload protection can be all used as handlers alongside each other:

config :my_app, :logger, [
  {:handler, :file_log, :logger_std_h, ...},
  {:handler, :sentry, Sentry.LoggerHandler, ...}
]

Whenever you want a custom logger but don’t want to implement overload protection, you can take advantage of LoggerBackends and write a custom logger backend instead.

In reality, I’ve yet to find myself writing a custom handler as the built-in :logger_std_h in combination with filters and formatters is very very flexible. So flexible, in fact, that LoggerJSON project was completely rewritten from being a logger backend to being a formatter.

Aaaaah, that makes sense! I could have implemented my example (file logger) using the logger_std_h handler indeed. So, the idea here is to rather configure the existing logger_std_h or logger_disk_log_h implementations unless you need something special, but which doesn’t need overload protection. In that case, you can use the LoggerBackends library. Thanks for the clarification!

Just for future reference, I managed to implement a logger_handler myself, but it’s terrible and shouldn’t be used except for inspiration for how to use e.g. :logger_formatter, how to convert the old format-option to the new template-option, or how to convert the gregorian microsecond timestamp of the new log_event back to date and time.

It’s on GitHub here: run-elixir/assets/archived_code/file_logger.ex at main · PJUllrich/run-elixir · GitHub

And I also implemented an “old” handler, which you should now add using the LoggerBackends.add/2 function instead of the deprecated Logger.add_backend/2 function. It’s on GitHub here: run-elixir/assets/archived_code/old_file_logger.ex at main · PJUllrich/run-elixir · GitHub

Interested in what you put together if you can share? I’m currently working on Logflare and we actually need beta testers for our new logger backend.

I really like being able to keep the raw event data as you can drill into it vs the time series approach. As long as your db is fast enough to query, you’re able to insert in real-time and you’re happy with the visualization / BI tools available … I think the time series approach is less favorable.

With our Logger backend you can structure logs and add any data you want. Logflare uses BigQuery as the db and manages the BigQuery schema so you can easily query each field specifically. And BigQuery plus Data Studio is very nice although I wish it would give me minutely graphs out of the box.