Filter sensitive logging data

config :phoenix, :filter_parameters, ["password", "secret"]

or check your logging service for equivalent..

Gotcha, but that’s just for incoming parameters correct? There’s no way to filter sensitive information logged during an error?

I have the same situation right now. I am making a request to a remote system and if it fails, I log the entire HTTPoison response, which might contain login credentials. Currently:

Body:
{:form, [{"email", "user@example.com"}, {"password", "secret1234"}]}

But maybe in the future, if the remote API accepts a JSON body, the body will be a JSON string. So something more general would be useful. A quick search on hex.pm didn’t yield any results.

can you post full (pseudo) code and error (sensitive data deleted..)

I imagine one has to filter manually - but maybe it’s possible to create/have a “deep” nested filter that goes through things..

eg
config :phoenix, :deep_filter_logs, [“password”, “secret”]

Basically, what I do is this:

defimpl AppGateway.Error, for: AppGateway.Errors.RemoteServiceError do
  def status(_), do: AppGateway.Errors.RemoteServiceError.status()

  def log_message(error) do
    """
    Error while calling Remote Service!

    Method: #{error.method}
    Url: #{error.url}

    Headers:
    #{inspect(error.headers)}

    Body:
    #{inspect(error.body)}

    Response:
    #{inspect(error.response)}

    Error:
    #{inspect(error.error)}
    """
  end
end

error.body is what I passed to HTTPpoison and error.response is what I get from the {:error, value} tuple from HTTPoison, which is in most cases a %HTTPoison.Response{} struct which also contains the full %HTTPoison.Request{} with the body.

Even manually filtering it is quite some work and the maintenance efforts are high. What I hoped was a clever algorithm that uses the string representation of the structs and filters out the data.

look at nested_filter nested_filter | Hex or deep_clean deep_clean | Hex

else a regex on the string containing sensitive data?

I think filtering the actual data is much easier and less error prone than an unstructured string dump of the data…

Seems like the nested_filter lib only works with maps but I am dealing with quite a few structs. Also, if the request body contains a JSON string, it won’t help at all.