neuone
Tesla.Middleware.JSON convert keys to lowercase
SCENARIO
I have this 3rd party API JSON that has it’s keys in TitleCase format. These API calls are all “GET” requests.
“MetaData” => %{
“UserName” => …
“LastName” => …
}%
I want to convert the keys from Titlecase to lowercase atom keys before I start to internally break down the json into internal structs.
HTTP Client
Tesla for my HTTP Client
Option 1 - Tesla Middleware JSON Engine Options
I was reading the Tesla.Middleware.JSON docs and I saw this option engine_opts: [keys: :atoms].
defmodule Api.Client.Base do
use Tesla
plug Tesla.Middleware.BaseUrl, "http://somewebsite.com"
plug Tesla.Middleware.JSON, engine_opts: [keys: :atoms]
But this converts my JSON from MetaData to metaData for the keys. Not ideal as I want them to be lowercase and underscore. So ideally meta_data
Option 2 - Build my own Middleware
I have never done this before but the instructions and documentation make it easy
defmodule Api.Client.InspectHeadersMiddleware do
@behaviour Tesla.Middleware
def call(env, next, _) do
env
|> Tesla.run(next)
|> decode_json()
end
def decode_json(env) do
with {:ok, tesla_struct} <- env,
{:ok, json} <- convert_to_json(tesla_struct),
results <- keys_to_underscore(json) do
{:ok, put_in(tesla_struct.body, results)}
end
end
def convert_to_json(strct) do
strct.body
|> Jason.decode()
end
def keys_to_underscore(json) when is_map(json) do
Map.new(json, &reduce_keys_to_underscore/1)
end
def reduce_keys_to_underscore({key, val}) when is_map(val), do: {string_to_atom(key), keys_to_underscore(val)}
def reduce_keys_to_underscore({key, val}) when is_list(val), do: {string_to_atom(key), Enum.map(val, &keys_to_underscore(&1))}
def reduce_keys_to_underscore({key, val}), do: {string_to_atom(key), val}
def string_to_atom(key) do
Macro.underscore(key)
|> String.to_atom
end
end
And then I make sure to place my middleware plug Api.Client.InspectHeadersMiddleware after the Tesla.Middleware.JSON
defmodule Api.Client.Base do
use Tesla
plug Tesla.Middleware.BaseUrl, "http://somewebsite.com"
plug Tesla.Middleware.JSON
plug Api.Client.InspectHeadersMiddleware
Conclusion
And this works but I have concerns and need some feedback.
QUESTIONS
1. Other possible solutions?
Has anyone had to deal with a similar situation? Any open source libraries I should check out?
2. The process of converting 3rd party JSON API keys and String.to_atom
I’m reading posts that suggest converting keys to atoms from 3rd party sources can be dangerous due to garbage collection limit String.to_atom, When to use Elixir String.to_atom and even this post
3. Is my solution putting me in danger
Is using a Middleware approach ideal OR is there something better?
The API requests are just GET requests and I do trust the service.
And I do know what the keys will be for each endpoint.
Thanks
Marked As Solved
stefanchrobot
Custom middleware that converts keys is the way to go. The idea behind middlewares was to provide an extension point. So something like:
defmodule Api.Client.Base do
use Tesla
plug Tesla.Middleware.JSON
# Once this middleware is called, the body is already deserialized from JSON.
plug KeyConversionMiddleware
end
would do the job.
I would suggest keeping the keys as strings, exactly for the reasons you’ve mentioned. Consider using Ecto schemas and changesets to validate the incoming requests and convert them to structs - it will convert the string keys for you.
Also Liked
neuone
I have done what you suggested and just kept the 3rd party api keys as strings but converted to “lowercase” and then through some module and then using Ecto.Schema to cast the string keys.
Helpful tip and good to know my approach is ideal.
Thanks for the feedback.
Wigny
I came across the same problem converting cases, so create a package to be able to reuse the approach in other projects: TeslaKeys.Middleware.Case — tesla_keys v0.3.0
neuone
Thanks. Will try this out in another project.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








