Prevent truncation of GenServer crashes

Questions / Help
, , ,
1

Hi,

I have a worker GenServer posting some results to its manager GenServer. The result is rather long and may contain hundreds of items. In some cases, this causes the manager GenServer to crash, however, I’m unable to see the stacktrace, because it gets truncated because of function argument size.

Here’s a shortened example:

02:17:00.222 [error] GenServer #PID<0.2821.0> terminating
** (stop) exited in: GenServer.call(Manager, {:post_results, %{data: ["2001167", "20011
68", "2001172", "2001169", "2001173", "2001176", "2001174", "2001175", "2001170", "20
01171", "2001178", "2001177", "2001196", "2001187", "2001188", "2001189", "2000489", "2001191", "2001192", "2001193", "2001194", "2001841", "202047
2", "2006815", "2019619", "2019942", "2020265", "2020588", "2019611", "201 (truncated)

This makes debugging the crashes very difficult. Is there a way to prevent the logs from truncating the crash data? Is there an alternative way of debugging such crashes?

Thank you!

2

You can set the :truncate option to an appropriate value:

:truncate - the maximum message size to be logged (in bytes). Defaults to 8192 bytes. Note this configuration is approximate. Truncated messages will have " (truncated)" at the end. The atom :infinity can be passed to disable this behavior.

5 Likes
3

Unfortunately, the truncate setting has no effect, I’ve already tried that. Here’s my logger config:

config :logger,
  backends: [
    :console,
    {LoggerFileBackend, :error},
  ]
config :logger, :error,
  path: "/home/me/logs/scraper/error.log",
  level: :error,
  handle_otp_reports: true,
  handle_sasl_reports: true,
  truncate: :infinity

config :logger, :console,
  handle_otp_reports: true,
  handle_sasl_reports: true,
  level: :debug,
  truncate: :infinity

Both console, and the file have the log messages truncated.

Am I perhaps missing something?

2 Likes
4

I have exact same need (to see which file and line blows up), my solution is inspecting inside terminate/2 callback.

# You will know it when you need it
def terminate({_, trace}, _socket) do
  IO.inspect(trace, label: "terminate trace")
end

It will show file and line number without data log.

3 Likes
5

Saved me hours. Thanks!