neuone

neuone

Tesla.Middleware.JSON convert keys to lowercase

SCENARIO

I have this 3rd party API JSON that has it’s keys in TitleCase format. These API calls are all “GET” requests.

“MetaData” => %{
“UserName” => …
“LastName” => …
}%

I want to convert the keys from Titlecase to lowercase atom keys before I start to internally break down the json into internal structs.

HTTP Client

Tesla for my HTTP Client

Option 1 - Tesla Middleware JSON Engine Options

I was reading the Tesla.Middleware.JSON docs and I saw this option engine_opts: [keys: :atoms].

defmodule Api.Client.Base do

  use Tesla
  plug Tesla.Middleware.BaseUrl, "http://somewebsite.com"
  plug Tesla.Middleware.JSON, engine_opts: [keys: :atoms]

But this converts my JSON from MetaData to metaData for the keys. Not ideal as I want them to be lowercase and underscore. So ideally meta_data

Option 2 - Build my own Middleware

I have never done this before but the instructions and documentation make it easy

defmodule Api.Client.InspectHeadersMiddleware do
  
  @behaviour Tesla.Middleware
  
  def call(env, next, _) do
    env
    |> Tesla.run(next)
    |> decode_json()
  end

  def decode_json(env) do
    with  {:ok, tesla_struct} <- env,
          {:ok, json} <- convert_to_json(tesla_struct),
          results <-  keys_to_underscore(json) do
      {:ok, put_in(tesla_struct.body, results)}
    end
  end


  def convert_to_json(strct) do
    strct.body
    |> Jason.decode()
  end

  def keys_to_underscore(json) when is_map(json) do
    Map.new(json, &reduce_keys_to_underscore/1)
  end

  def reduce_keys_to_underscore({key, val}) when is_map(val), do: {string_to_atom(key), keys_to_underscore(val)}
  def reduce_keys_to_underscore({key, val}) when is_list(val), do: {string_to_atom(key), Enum.map(val, &keys_to_underscore(&1))}
  def reduce_keys_to_underscore({key, val}), do: {string_to_atom(key), val}

  def string_to_atom(key) do
    Macro.underscore(key)
    |> String.to_atom
  end

end

And then I make sure to place my middleware plug Api.Client.InspectHeadersMiddleware after the Tesla.Middleware.JSON

defmodule Api.Client.Base do

  use Tesla
  plug Tesla.Middleware.BaseUrl, "http://somewebsite.com"
  plug Tesla.Middleware.JSON
  plug Api.Client.InspectHeadersMiddleware

Conclusion

And this works but I have concerns and need some feedback.

QUESTIONS

1. Other possible solutions?
Has anyone had to deal with a similar situation? Any open source libraries I should check out?

2. The process of converting 3rd party JSON API keys and String.to_atom
I’m reading posts that suggest converting keys to atoms from 3rd party sources can be dangerous due to garbage collection limit String.to_atom, When to use Elixir String.to_atom and even this post

3. Is my solution putting me in danger
Is using a Middleware approach ideal OR is there something better?
The API requests are just GET requests and I do trust the service.
And I do know what the keys will be for each endpoint.

Thanks

Marked As Solved

stefanchrobot

stefanchrobot

Custom middleware that converts keys is the way to go. The idea behind middlewares was to provide an extension point. So something like:

defmodule Api.Client.Base do
  use Tesla
  plug Tesla.Middleware.JSON
  # Once this middleware is called, the body is already deserialized from JSON.
  plug KeyConversionMiddleware
end

would do the job.

I would suggest keeping the keys as strings, exactly for the reasons you’ve mentioned. Consider using Ecto schemas and changesets to validate the incoming requests and convert them to structs - it will convert the string keys for you.

Also Liked

neuone

neuone

I have done what you suggested and just kept the 3rd party api keys as strings but converted to “lowercase” and then through some module and then using Ecto.Schema to cast the string keys.

Helpful tip and good to know my approach is ideal.

Thanks for the feedback.

Wigny

Wigny

I came across the same problem converting cases, so create a package to be able to reuse the approach in other projects: TeslaKeys.Middleware.Case — tesla_keys v0.3.0

neuone

neuone

Thanks. Will try this out in another project.

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&amp;query=perfume&amp;page=2, I would like to get: ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New

Other popular topics Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Qqwy
Update: How to use the Blogs &amp; Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 126479 1222
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement