Logging to disk with rotation

Questions / Help
, ,
1

In my Erlang application, I have the following configuration in app.config:

[
{kernel,
  [{logger,
    [
       %% Console logger
       {handler, default, logger_std_h, #{}},
       %% Disk logger for errors
       {handler, disk_log, logger_disk_log_h, #{config => #{
            file => "log/web_server.log",
            type => wrap,
            max_no_files => 5,
            max_no_bytes => 2097152,
            sync_mode_qlen => 2000, % If sync_mode_qlen is set to the same value as drop_mode_qlen,
            drop_mode_qlen => 2000, % synchronous mode is disabled. That is, the handler always runs
            flush_qlen => 5000,     % in asynchronous mode, unless dropping or flushing is invoked.
            overload_kill_enable => true
            % Documentation about Overload protection, together with default values, can be found here:
            % http://erlang.org/doc/apps/kernel/logger_chapter.html#protecting-the-handler-from-overload
         },
         formatter => {logger_formatter, #{
            depth => 12,
            chars_limit => 1024
          }}
        }
       }
    ]}
]},
...]

I have tried to find the relevant documentation explaining how I can configure the Logger in my Elixir config files to achieve the above behaviour (or similar) for my Phoenix application. I have looked here: Logger — Logger v1.13.4
But there is no information to be found. Does anyone know where I can find an example config for disk logging in Elixir ?

2 Likes
2

You can translate the config above to Elixir’s config/config.exs or config/runtime.exs. Here is the first handler as an example:

config :kernel, :logger, [
  {:handler, :default, :logger_std_h, %{}}
]

You can also add additional handlers by calling :logger.add_handler and friends in your application start callback.

2 Likes
3

Thank you for your rapid answer.

I added these lines to config/config.exs:

config :kernel, :logger, [
  {:handler, :default, :logger_std_h, %{}},
  {:handler, :disk_log, :logger_disk_log_h,
    %{
      config: %{
            file: "log/my_app.log",
            type: :wrap,
            max_no_files: 5,
            max_no_bytes: 2097152,
            sync_mode_qlen: 2000, # If sync_mode_qlen is set to the same value as drop_mode_qlen,
            drop_mode_qlen: 2000, # synchronous mode is disabled. That is, the handler always runs
            flush_qlen: 5000,     # in asynchronous mode, unless dropping or flushing is invoked.
            overload_kill_enable: true
            # Documentation about Overload protection, together with default values, can be found here:
            # http://erlang.org/doc/apps/kernel/logger_chapter.html#protecting-the-handler-from-overload
      },
      formatter: {:logger_formatter, %{depth: 12, chars_limit: 1024}}
    }
  }
]

When I add these lines lines to the config file, however, and I start the application with iex -S mix phx.server I get the following error message:

Cannot configure base applications: [:kernel]

These applications are already started by the time the configuration
executes and these configurations have no effect.

If you want to configure these applications for a release, you can
specify them in your vm.args file:

    -kernel config_key config_value

Alternatively, if you must configure them dynamically, you can wrap
them in a conditional block in your config files:

    if System.get_env("RELEASE_MODE") do
      config :kernel, ...
    end

and then configure your releases to reboot after configuration:

    releases: [
      my_app: [reboot_system_after_config: true]
    ]

This happened when loading config/config.exs or
one of its imports.

So I added rel/vm.args.eex to the project. I am currently trying to find the right syntax to pass the logger config there. Is this the right way to go ?

4

Ah, that’s true, I forgot about this detail. I believe we will be able to address this in future Elixir releases since a feature has been added to Erlang/OTP.

So I think calling :logger.add_handler in your application start callback is the way to go!

3 Likes
5

Thank you for your help. I managed to make it work by adding the handler as you said.

For anyone who wants to achieve the same. In your config/config.exs define the disk logging configuration by adding these lines (with my_app replaced by the name of your application):

config :my_app, :logger, [
  {:handler, :disk_log, :logger_disk_log_h, %{
      config: %{
            file: 'log/my_app.log',
            type: :wrap,
            max_no_files: 5,
            max_no_bytes: 2097152,
            sync_mode_qlen: 2000, # If sync_mode_qlen is set to the same value as drop_mode_qlen,
            drop_mode_qlen: 2000, # synchronous mode is disabled. That is, the handler always runs
            flush_qlen: 5000,     # in asynchronous mode, unless dropping or flushing is invoked.
            overload_kill_enable: true
            # Documentation about Overload protection, together with default values, can be found here:
            # http://erlang.org/doc/apps/kernel/logger_chapter.html#protecting-the-handler-from-overload
      },
      formatter: {:logger_formatter, %{depth: 12, chars_limit: 1024}}
    }
  }
]

(Notice the single quotes for the file field. We need an Erlang string here. It won’t be able to read the field otherwise.)

Then in lib/my_app/application.ex (in a phoenix project) add the logger by adding :logger.add_handlers(:my_app):

def start(_type, _args) do
  :logger.add_handlers(:my_app)
  ...
end

This should do it.

Thanks, José, for indicating the solution.

3 Likes
6

Glad to help @JeyHey!

Could you submit a PR to the logger documentation explaining how to add this as an example? I think it could be helpful to others in the future.

3 Likes
7

I’ll take a look at it and try to do so. Happy to contribute : ). It can take a few days, though, as it would be the first time for me to make a pull request to the documentation.

1 Like
8

Small note - it is better to use :logger_std_h which also supports logging to disk (type: file, file: 'path'). It behave IMHO more reasonable than :logger_disk_log_h. Namely it will always log new messages to path while :logger_disk_log_h does some weird rotation where you need to check which file it is currently writing to.

2 Likes