gavid

gavid

Recursively replace all JSON `$ref`s with the code that they refer to

In JSON, it is possible in some formats (e.g. OpenAPI, React-JSON-Schema-Forms etc.) to use the $ref keyword as a rough parallel of macro functionality (see here: JSON References | NIEM GitHub).

This creates an issue when decoding JSON to Elixir (e.g. with Poison or Jason). I may be processing a complex JSON structure, and instead of the data I need being under "someKey", I have "$ref": "/path/to/something".

What I need is a way to replace all $refs with the actual JSON data that they represent. I have no idea how to do this, especially since $refs can be nested (i.e. a $ref refers to a JSON object with other refs inside it etc.). Also, in OpenApi, it is possible to add a attributes for summary and description after the $ref, but it is not possible to add any other attributes (How to use JSON references ($refs)).This creates further parsing complexity.

Is there some existing library that I can use maybe?

Marked As Solved

codeanpeace

codeanpeace

One approach would be preprocessing the JSON with a JS/TS library like json-schema-ref-parser to de-reference/resolve the references before decoding it in Elixir.

If you really wanted to do this in Elixir, I’d take a look at how the JS libraries handle tree traversal and re-implement it in Elixir. I imagine there would be opportunities for recursion.

Off the top of my head, Elixir’s built-in get_in/2 and update_in/3 functions in the Kernel module paired with the Access module will be helpful. There’s also a Pathex library that could be a good alternative for working with nested data structures.

Edit to add: this github issue might be relevant
https://github.com/open-api-spex/open_api_spex/issues/517

Also Liked

al2o3cr

al2o3cr

Both of the examples you linked to are specifically about embedding references in schemas (either YAML- or JSON-formatted) to reduce duplication; I don’t believe a normal application decoding plain JSON would ever encounter a $ref outside of that context.

If you are expecting input like this, make sure to be careful about which URLs you follow and when. For instance, if you wrote a JSON parser that loaded these references before validating the shape of the data you could accidentally create a denial-of-service reflector. For instance, imagine a malicious payload like:

{
  "foo1": {
    "$ref": "http://example.com/oh_no1#foo"
  },
  "foo2": {
    "$ref": "http://example.com/oh_no2#foo"
  },
  "foo3": {
    "$ref": "http://example.com/oh_no3#foo"
  },
  "foo4": {
    "$ref": "http://example.com/oh_no4#foo"
  },
  ...etc for thousands of times
}

A single request to an API that tries to “follow” all those refs would be multiplied into thousands pointed at example.com.

Where Next?

Popular in Questions Top

New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

We're in Beta

About us Mission Statement