williamthome

Exneus - JSON parser, generator and formatter for Elixir built on top of the new OTP json module

Exneus is an incredibly flexible and performant JSON parser, generator and formatter for Elixir. It is a wrapper for Elixir of Euneus, an Erlang library built on the top of the new OTP json module.

Both encoder and decoder fully conform to RFC 8259 and ECMA 404 standards and are tested using JSONTestSuite.

The new OTP json module was introduced in OTP-27, which is compatible with Elixir v1.17, but by installing json_polyfill, the minimum requirement reduces to OTP-24 and Elixir v1.12. I’ve not tested it in other versions, but the CI says it works.

Exneus/Euneus’s goal is to be very flexible and performant.

The first release of Exneus is an Erlangish lib. I just adapted Euneus’s Erlang code to Elixir. This will change

Further explanation and examples are available at hexdocs.

Features

Encode
Decode
Stream
Minify
Format

Encode option details

codecs - Act like plugins. There are some built-in options, and it is also possible to create custom ones.

Currently built-in encode codecs:
- timestamp
- datetime
- ipv4
- ipv6
- records
codec_callback - - Overrides the default codec resolver.
nulls - Defines which values should be encoded as null.
skip_values - Defines which map values should be ignored. This option permits achieves the same behavior as Javascript, which ignores undefined values of objects.
key_to_binary - Overrides the default conversion of map keys to a string.
sort_keys - Defines if the object keys should be sorted.
keyword_lists - If true, converts keyword_lists into objects.
escape - Overrides the default string escaping.
encode_integer - Overrides the default integer encoder.
encode_float - Overrides the default float encoder.
encode_atom - Overrides the default atom encoder.
encode_list - Overrides the default list encoder.
encode_map - Overrides the default map encoder.
encode_tuple - Overrides the default tuple encoder.
encode_pid - Overrides the default pid encoder.
encode_port - Overrides the default port encoder.
encode_reference - Overrides the default reference encoder.
encode_term - Overrides the default encoder for unsupported terms, like functions.

Decode option details

codecs - Act like plugins. There are some built-in options, and it is also possible to create custom ones.

Currently built-in decode codecs:
- timestamp
- datetime
- ipv4
- ipv6
- pid
- port
- reference
null - Defines which term should be considered null.
binary_to_float - Overrides the default binary to float conversion.
binary_to_integer - Overrides the default binary to integer conversion.
array_start - Overrides the :json.array_start_fun/0 callback.
array_push - Overrides the :json.array_push_fun/0 callback.
array_finish - Overrides the :json.array_finish_fun/0 callback.

In addition to the custom function, there are:
- ordered
- reversed
object_start - Overrides the :json.object_start_fun/0 callback.
object_keys - Transforms JSON objects key into Elixir term.

In addition to the custom function, there are:
- binary
- copy
- atom
- existing_atom
object_push - Overrides the :json.object_push_fun/0` callback.
object_finish - Overrides the :json.object_finish_fun/0 callback.

In addition to the custom function, there are:
- map
- keyword_list
- reversed_keyword_list

Benchmark

The next step is to improve the performance since no optimization has been done yet. However, there is a simple benchmark comparing it with some known libraries, including the OTP json module.

IMPORTANT!

The benchmark result below just gives an idea of performance since the benchmark runs are very simplistic. Subsequent releases will update those values.

Also, to be an honest benchmark, all libs should produce the same output. I have not checked this.

Encode results:

Name                 ips
euneus           36.95 K
json (OTP)       35.85 K - 1.03x slower +0.83 μs
Exneus           35.71 K - 1.03x slower +0.94 μs
jiffy            34.70 K - 1.06x slower +1.76 μs
Jason            26.75 K - 1.38x slower +10.32 μs
thoas            16.40 K - 2.25x slower +33.92 μs
jsone            15.52 K - 2.38x slower +37.38 μs
jsx               4.21 K - 8.77x slower +210.29 μs
JSON              3.88 K - 9.52x slower +230.47 μs

Decode results:

Name                 ips
json (OTP)       18.79 K
euneus           18.06 K - 1.04x slower +2.16 μs
Exneus           17.68 K - 1.06x slower +3.34 μs
Jason            15.16 K - 1.24x slower +12.74 μs
jsone            13.82 K - 1.36x slower +19.17 μs
jiffy            12.76 K - 1.47x slower +25.15 μs
thoas            11.89 K - 1.58x slower +30.89 μs
jsx               4.93 K - 3.81x slower +149.64 μs
JSON              2.43 K - 7.74x slower +358.48 μs

If you liked those tools, find them helpful, and would like to see them improved, please consider sponsoring me or buying me some coffees

Feel free to submit suggestions and to contribute

https://github.com/williamthome/exneus

10 comments

#erlang #hex #library #otp #json #github #exneus #euneus

21 662 10

2024-08-10 06:16:34 UTC

Most Liked

williamthome

Sorry, I wasn’t clear.

Using just the OTP :json module without any other lib is fine and is extremely fast! It provides encoding and decoding:

iex(1)> :json.encode(:foo)
[34, "foo", 34]
iex(2)> :json.decode("\"foo\"")
"foo"

However, the OTP :json module is available starting from OTP-27 and provides few options. On the other hand, :euneus is built on top of :json, providing more flexibility and options, including a JSON formatter.

For example, to decode Map keys as atoms:

iex(1)> :json.decode(~s({"foo":"bar"}), [], %{object_push: fn k, v, acc -> [{String.to_atom(k), v} | acc] end})
{%{foo: "bar"}, [], ""}
iex(2)> :euneus.decode(~s({"foo":"bar"}), %{object_keys: :atom})
%{foo: "bar"}

And that’s just one example. Also, I adapted the OTP json module into a lib called json_polyfill to increase the compatibility with OTP and Elixir versions, OTP-24 and Elixir v1.12, respectively.

Please let me know if this answers your question.

Post #5

D4no0

Quick question, why erlang devs need Euneus and can’t use the json library directly from OTP?

Post #2

williamthome

Thiago, I truly appreciate your perspective and respect your comment. Not only for Elixir, but Michał is also an extremely important person to the Erlang community, and his contributions are always great!

I do not know exactly what you mean by “fragmentation.”. Don’t get me wrong too, but I’m not inclined to start a discussion here comparing Exneus to another existing library. The main points of Exneus are in the thread.

Each library has its strengths and weaknesses. If you find a library that addresses your project’s requirements, feel free to consider using it. At the end of the day, libraries are tools, and choosing one over another is entirely optional. Moreover, libraries can always learn from and benefit from one another, and I would be delighted if Exneus could bring value to others.