Sensitive data in stacktrace

tim2CF

Let’s say we have a module like

defmodule Foo do
  @salt Application.get_env(:foo, :salt, [])
  def hash(password) do
    try do
      {:ok, md5(password, @salt)}
    rescue 
      e -> {:error, {e, __STACKTRACE__}}
    end
  end
  defp md5(password, salt = [_ | _]) do
    Enum.reduce(salt, password, &:erlang.md5(&2<>&1))
  end
end

And when we call &hash/1 function it returns error

iex(19)> Foo.hash "123"
{:error,
 {%FunctionClauseError{
    args: nil,
    arity: 2,
    clauses: nil,
    function: :md5,
    kind: nil,
    module: Foo
  },
  [
    {Foo, :md5, ["123", []], [file: 'iex', line: 27]},
    {Foo, :hash, 1, [file: 'iex', line: 22]},
    {:erl_eval, :do_apply, 6, [file: 'erl_eval.erl', line: 677]},
    {:elixir, :eval_forms, 4, [file: 'src/elixir.erl', line: 265]},
    {IEx.Evaluator, :handle_eval, 5, [file: 'lib/iex/evaluator.ex', line: 249]},
    {IEx.Evaluator, :do_eval, 3, [file: 'lib/iex/evaluator.ex', line: 229]},
    {IEx.Evaluator, :eval, 3, [file: 'lib/iex/evaluator.ex', line: 207]},
    {IEx.Evaluator, :loop, 1, [file: 'lib/iex/evaluator.ex', line: 94]}
  ]}}

And we want to put this stacktrace to Logger (let’s say for easier debugging)
But we see, what stacktrace can contain sensitive data (in example is password = “123”)
Of course we can remove all arguments from stacktrace manually, but in most cases arguments are useful
Maybe there is already implemented solution to remove some arguments of some functions from stacktrace by marking them in module definition somehow?

It would be nice to say somehow that n-th argument of foo function in module Bar is sensitive and should be excluded from stacktrace

4 comments

#stacktrace

1 3275 4

2022-09-15 09:32:05 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

sfusato

There is ‘Custom struct inspections’ that was just released with Elixir 1.8:
https://github.com/elixir-lang/elixir/blob/v1.8/CHANGELOG.md

Maybe return just the tail of the stack trace and at an upper-level inspect your struct with the returned stack trace. This would obfuscate the password field or whatever you set to hide from your struct.

Post #1

josevalim

Creator of Elixir

Here is what Plug.Crypto does regarding the stacktrace: plug_crypto/lib/plug/crypto/key_generator.ex at 76a595f26529cd550f32166c3288358e1785fb1f · elixir-plug/plug_crypto · GitHub

Post #2

zazaian

@sfusato’s response is very good, though it seems that in some instances you’d want to expressly name the excluded struct parameters, rather than list all of the unexcluded ones.

The Inspect protocol actually allows you to name the struct attributes that are excluded from inspection, which is cleaner for circumstances in which most of a struct’s attributes are not private or secret. Here’s an example:

defmodule CustomRequest do
  @derive {Inspect, except: [:password]}
  defstruct [:name, :email, :password]
end

If you had a myriad of secret attributes within a single struct though, using only is probably cleaner and safer, as adding additional secret attributes would not require you to also add them for exclusion:

defmodule CustomRequest do
  @derive {Inspect, only: [:name, :email]}
  defstruct [
    :name,
    :email,
    :password,
    :secret_token,
    :secret_id
  ]
end

Please note that the @derive call does have to be declared above the defstruct keyword as indicated in the Inspect docs, as shown above.

Post #3

chocolatedonut

Despite all the helpful tips, I still am not able to hide password of the following :gen_statem-based RabbitMQ consumer:

defmodule Rabbit.Consumer do
  @derive {Inspect, only: [:channel]}
  defstruct [:config, :channel]
end

I’d like that the :config field (above), which includes the password, be hidden in all these cases:

:sys.get_state Rabbit.Consumer
when Rabbit.Consumer outright crashes and produces such stacktrace (observe password in plain sight)

[error] CRASH REPORT Process 'Rabbit.Consumer' with 0 neighbours exited with reason: 
  no such process or port in call to 
  gen_server:call(<0.1026.0>, {call,{'basic.ack',1,false},none,<0.1012.0>}, 15000) 
  in gen_server:call/3 line 382

[error] Supervisor 'Rabbit.Application' had child 'Rabbit.Consumer' started with
 'Rabbit.Consumer':start_link([
   {host,<<"localhost">>},
   {username,<<"admin">>},
   {password,<<"admin">>},
   {virtual_host,<<"/">>},
   ...]) 
   at <0.3177.0> exit with reason 
   {shutdown,{gen_server,call,[<0.3196.0>,
     {call,{'queue.declare',0,nil,false,true,false,false,false,[]},none,<0.3177.0>},15000]}} 
   in context child_terminated

What I tried already

`format_status`

I added :gen_statem c:format_status/2 and also OTP 25 c:format_status/1:

def format_status(s) do  # format_status/1 in OTP 25
  Map.drop s.data, [:config]
end

This did help as :sys.get_status didn’t print the sensitive :config anymore. But, it didn’t help with neither of the cases 1. or 2. above.

Prune args from stacktrace

I wrapped Rabbit.Consumer’s call to a different process with a rescue and pruned args from System.stacktrace() (using the suggested Plug.Crypto.prune_args_from_stacktrace/1). But, this doesn’t help when :gen_statem/Rabbit.Consumer itself crashes. How can one prune_args_from_stacktrace of the process itself (be it GenServer, :gen_statem, etc.)?

`Process.flag(:sensitive, true)`

def init(opts) do
  Process.flag(:sensitive, true)
  # etc.

Neither of the cases 1. or 2. above were solved.

Docs for the :sensitive flag say

Sets or clears flag sensitive for the current process. When a process has been marked as sensitive by calling process_flag(sensitive, true), features in the runtime system that can be used for examining the data or inner working of the process are silently disabled.

Features that are disabled include (but are not limited to) the following:

Tracing. Trace flags can still be set for the process, but no trace messages of any kind are generated. (If flag sensitive is turned off, trace messages are again generated if any trace flags are set.)

Sequential tracing. The sequential trace token is propagated as usual, but no sequential trace messages are generated.

process_info/1,2 cannot be used to read out the message queue or the process dictionary (both are returned as empty lists).

Stack back-traces cannot be displayed for the process.

In crash dumps, the stack, messages, and the process dictionary are omitted.

If {save_calls,N} has been set for the process, no function calls are saved to the call saving list. (The call saving list is not cleared. Also, send, receive, and time-out events are still added to the list.)

The following bolded part from above

“Features that are disabled include (but are not limited to)”.

made me hopeful that state will also be included (among disabled features).

So, I saw the following on erlef.github.io:

“Process state […] cannot be introspected”

Alas, this doesn’t seem to be true - :sys.get_state Rabbit.Consumer still shows the password.

Here’s this section verbatim:

Finally, a process can be marked as ‘sensitive’, using erlang:process_flag/2. This has the following effect:

Message queue contents cannot be introspected, and is not written to a crash dump

Process dictionary cannot be introspected, and is not written to a crash dump

Process state of a gen_server, gen_event or gen_statem process cannot be introspected, and is not written to a crash dump

Process heap and stack are not written to a crash dump

The process cannot be traced