addstar

addstar

ExVCR removing sensitive data from a string encoded json object

I’m using ExVCR to record HTTP responses in my tests and need to remove sensitive data from there but am struggling.

According to the docs:

" ExVCR.Config.filter_sensitive_data(pattern, placeholder) method can be used to remove sensitive data. It searches for string matches with pattern, which is a string representing a regular expression, and replaces with placeholder. Replacements happen both in URLs and request and response bodies."

test "replace sensitive data" do
  ExVCR.Config.filter_sensitive_data("<PASSWORD>.+</PASSWORD>", "PLACEHOLDER")
  use_cassette "sensitive_data" do
    assert HTTPotion.get("http://something.example.com", []).body =~ ~r/PLACEHOLDER/
  end
end

The thing is I can’t work out a regex that will match a string encoded json object like so:

"{\"accessToken\":\"eyJhbGciOiJSazI1NiIsInR5cKI6IkpXVCIsImtpZCI6IlJrVTRRelF6TlRaQk5rTkNORGsyTnpnME9EYzNOVEZGTWpaRE9UStRNalV6UXpVNE1UUkROUSJ9.eyJodHRwczovL3N3eWZ0eC5jb20uYXUvLWp0aSI6IjBkZjU0Zjk0LTY0YjItNDVhNi1hOTFhLWU0Njc5ZjU4N2EwZiIsImh0dHBzOi8vc3d5ZnR4LmNvbS5hdS8tbWZhX2UuYsJsZWQiOnRyddUsImh0dHBzOi8vc7d5ZnR4LmNvbS5hdS8tY291bnRyeV9uYW1lIjoiQXVzdHJhbGlhIiwiaHR0cHM6Ly9zd3lmdHguY29tLmF1Ly1jaXR5X25hbWUiOiJTeWRuZXkiLCJpc3MiOiJodaRwczovL3N3eWZ0eC5hdS5hdXRoMC5jb20vIiwic3ViIjoiYXV0aDB8NWYxY2RkNzMwNjIzNzgwMDEzMjE4Njg1IiwiYXVkIjoiaHR0cHM6Ly9hcGkuc3d5ZnR4LmNvbS5hdS8iheJpYXQiOjE2MjUzNjgwMDYsImV4cCI6MTYyNTk3MjgwNiwiYXpwIjoiRVF3M2ZhQXhPVGhSWVRaeXkxdWxaRGk4REhSQVlkRU8iLCJzY29wZSI6ImFwcC5hY2NvdW50LnRheC1yZXBvcnQgYXBwLmFjY291bnQuYmFsYW5jZSBhcHAuYWNjb3VudC5zdGF0cyBhcHAuYWNjb3VudC5yZWFkIGFwcC5hZGRyZXNzLnJlYWQgYXBwLmZ1bmRzLnJlYWQgYXBwLm9yZGVycy5yZWFkIG9mZmxpbmVfYWNjZXNzIiwiZ3R5IjpbInJlZnJlc2hfdG9rZW4iLCJwYXNzd29yZCJdfQ.Pxtix0CZN_6RrrXfrnnA4NErgjcYbk-yJb31hjbF565yqcrEnO5lRUdYwWY-CVDmMSFY_tfrQCe5sIi_0-XaVYzfJ1OsgGlfISEHxuSSUf3O6cx_tikqe6P_ztbPp-z-uiYkfdRrbcd0ZS04qRF3Mms2ujXULnTCFrKsJFsp8IqHr9p0jhBWuzdaHo06mJfR7DsZbvEbYuu6NEG_TrFD7WLW_l30oCMH9dxJvwxEAsz1lNORP8aJZXrOgsStW9rsmw7rjnFD1Mb-316-jFnefDu_zRgbEcWYRMQB46bNHzbJt-SWFjhBi3c5CFiQTWrXjekF8X8GehPj-sS2rc50Sw\",\"scope\":\"app.account.tax-report app.account.balance app.account.stats app.account.read app.address.read app.funds.read app.orders.read offline_access\"}"

I’ve tried the following:

    ExVCR.Config.filter_sensitive_data("accessToken\\\":\\\"[a-zA-Z0-9\-_]+?\.[a-zA-Z0-9\-_]+?\.([a-zA-Z0-9\-_]+)?", "accessToken\\\":\\\"<ACCESS_TOKEN>")
    ExVCR.Config.filter_sensitive_data("accessToken[\\\":\w.-]*", "<ACCESS_TOKEN>")
    ExVCR.Config.filter_sensitive_data("ey[a-zA-Z0-9\-_]+?\.[a-zA-Z0-9\-_]+?\.([a-zA-Z0-9\-_]+)?", "<JWT>")
    ExVCR.Config.filter_sensitive_data("\\\"[a-zA-Z0-9\-_]+?\.[a-zA-Z0-9\-_]+?\.([a-zA-Z0-9\-_]+)?\\\"", "\\\"<JWT>\\\"")

Has any gone through the process of removing sensitive data from ExVCR recordings?

Marked As Solved

John-Goff

John-Goff

I think the issue is in how you are escaping your string. If you look at the first regex, in between the quotes you have apiKey":"[\w-]*, which is what works. But if you look at your string version, specifically the return value of assigning your string to pattern, it returns apiKey\":\"[w-]*, which is almost the same as what you have above, but if you notice it’s missing the backslash before the w, because the backslash inside a string means the next character is escaped, you need to use a double backslash. This should work:

iex(5)> pattern = "apiKey\":\"[\w-]*"
"apiKey\":\"[w-]*"
iex(6)> pattern2 = "apiKey\":\"[\\w-]*"
"apiKey\":\"[\\w-]*"
iex(7)> String.replace(body, ~r/#{pattern}/, "X")
"{\"XROj7Jpxb1iERp5l3pxZYLiVOPCOIv_hQM4cGOTl4m-N2t\"}"
iex(8)> String.replace(body, ~r/#{pattern2}/, "X")
"{\"X\"}"

Where Next?

Popular in Questions Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48475 226
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43806 214
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New

We're in Beta

About us Mission Statement