Solid log aggregation setup?

Questions / Help
7

@pdilyard, plug_logger_json is only for plug projects but I ship other logs to ELK in the same way (json). I usually create a logger module in the project and define some log functions for some common logging use cases. It’s useful for everything to be JSON so that I can view all the logs with a shared field like request_id.

Here’s an example for logging outside of plug to ELK where I log external requests so I can trace a slow request and see if an external call is slowing things down. You’d probably want to do something similar for background logging. Just pick a different log_type other than http which is what plug_logger uses.

  def http(func, url, start) do
    _ = Logger.log :info, fn ->
      stop = :os.timestamp()
      %{
        "duration"        => Float.round(:timer.now_diff(stop, start) / 1000, 3),
        "function"        => func,
        "level"           => "info",
        "log_type"        => "external_request",
        "request_id"      => Logger.metadata[:request_id],
        "url"             => url
      }
      |> :jiffy.encode
    end
  end
2 Likes