Slowness issue due to logger_file_backend

Those settings are not for performance optimization. They exist to make sure your app doesn’t fall over when there’s to many logs to be handled. It does so by first making sure logs are written synchronously (app waits on logs to be written, making it write less logs) and if that doesn’t help by not writing logs at all (even less than before). Under normal operations you don’t want to hit those thresholds.

Not having a performance hit is not possible. Writing to disk is by definition slower than not needing to interact with the disk. I can’t speak for if the performance you’re seeing is reasonable or not, but it’s for sure expected to exist.

You can look into other ways to handle logs, which don’t require writing to disk. You might need to consider your constraints in terms of durability of logs though.

Not necessarily but I haven’t looked under the covers of Erlang’s or Elixir’s logging. I authored a small Golang logging wrapper some months ago (for sending log events to OpenObserve.AI) and I just had it spawn a background goroutine (== Elixir process) and a logger call was just sending message to its channel, and that data was sent to the network on arrival of that channel.

There are ways to do this better and I would be somewhat puzzled if Erlang / Elixir haven’t went about it the same way. I should probably go check the code.

Was this golang library writing to disk? It doesn’t sound like it did.

No, but sending to network can be even slower. Point is, the I/O was relegated to a background worker and was never happening in-place at the time of the logger calls.

That’s what the logger does by default as well as far as I know. But it switches to be synchronous or dropping logs when the backend gets overloaded.

That makes sense. Do you think that is what is happening to OP? Does not seem to be the case but who knows, we don’t know the entire story.

If you can update your Elixir and OTP versions, Elixir v1.15 comes with built-in support for file logging: Logger — Logger v1.20.2

How much log are we talking about, 1MB per second, 10MB per second, or 100MB per second?

I only log to console and use a logging daemon such as svlogd, which should be faster than the built-in file logger, but at very high logging throughput, you may need some custom solution.

Yes, that is the case happening with my application. The log traffic is huge which is getting piled up in Logger queue and after a sync_threshold its going into synchronous mode. I tried playing with the sync_threshold and discard_threshold settings but its not helping much after enabling file logging.

Thanks for the quick reply @josevalim. I tried upgrading the Elixir version to v1.15 and setting the deafult handler config as mentioned in the doc:

config :logger, :default_handler,
  level: :info,
  config: [
    file: ~c"system.log",
    filesync_repeat_interval: 5000,
    file_check: 5000,
    max_no_bytes: 10_000_000,
    max_no_files: 5,
    compress_on_rotate: true
  ]

config :logger, :default_formatter,
  format: "\n$time $metadata[$level] $message\n"

I agree that it gives the inbuilt functionality to log it into the file and removes the dependency of using logger_file_backend but its behaviour is almost similar to the logger_file_backend ie. it has overload protection.
Is there any way where we can keep this Logger constantly in :async mode. And what will be the consequences of doing it so?
Regards