Menkir

Menkir

Jexon - Lossless encoding to JSON

Hi everyone,

I recently released a small library built on top of Jason that converts Elixir maps and structs to JSON without loss. Atoms and tuples are data types that have no direct representation in JSON. As a result, keys and values of these data types are specially encoded with prefixes. Additionally, structs without the @derive attribute can be directly encoded to JSON.

The library offers three functions:

  • to_map: This function formats structs into maps and stores the __struct__ key.
  • to_json
  • from_json

Feel free to contribute in case you have more edge cases to consider.

Happy Coding!

Most Liked

adamu

adamu

Sorry to be that guy, but if the purpose is to have kind of transport encoding with the objective of eventual decoding back into Elixir, it seems that external term format would be a good choice, unless you need JSON for other interoperability reasons. Maybe some suggested use cases or a mention of this in the docs would be useful :slight_smile:

Menkir

Menkir

I fully understand this. We initially considered this formatting, but we had several concerns:

If you want to transport the state via JSON (for instance, for persistence), it’s necessary to encode it into Base64. Subsequently, you can store it in any DB of your choice. Yet, when you try to retrieve this state, you’re confronted with binary data, which is incomprehensible to both the remote machine and to you.

With Jexon-encoded Elixir structs, it becomes feasible to perform diffs between varied states or carry out migrations when certain keys have changed.

Debugging is more straightforward than when using ETF. I can effortlessly copy and paste parts or the entire state into a remote IEX, then pipe it through functions to diagnose the issue.

Ultimately, the decision boils down to preference: Would you rather store your data in binary or JSON format? In my specific situation, the data was initially stored locally in ETS. However, as is often the case, requirements shifted, necessitating the state’s transfer to a centralized backup service that exclusively accepts JSON.

mayel

mayel

Looks interesting! Some examples of the JSON output would be useful in the readme.

Last Post!

DEvil0000

DEvil0000

The issues with base64 or similar things are:

  • you can not diff on it naive and natively
  • it is less compactable (“zip”, double delta compression, …)
  • you need base64 encoding/decoding as a extra step when transport as json is given
  • not readable in native format

This also applies to binary formats or other codings.
JSON has in addition the advantage that basically all modern databases and other sotftware supports it natively.

As so often in software there are pros and cons and it depends on your needs if the approach fits. Of course there are other options and pathes one can choose for implementation or architecture.

Where Next?

Popular in Announcing Top

kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
pkrawat1
Hey guyz We at @aviabird are working on a payment library in elixir/phoenix. We are targeting March 2018 to add 56 Gateways to it. Have...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 30486 241
New
tmbb
I’ve decided to create this topic to discuss optimization possibilities for something like Phoenix LiveView. I’ve created this topic unde...
144 10847 141
New
devonestes
Introducing assertions, the library that helps you write really great test assertions! GitHub: GitHub - devonestes/assertions: Helpful a...
New
tfwright
After working on it for a couple of months and using it in production for most of that time, today I’ve released LiveAdmin, a LiveView ba...
New
Flo0807
Hello everyone! I am excited to share our heart project Backpex with you. After building several Phoenix applications, we realized that...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement