coby

coby

JSON Library Benchmarks

Hi Everyone!

First time writer, long time reader here.

I’ve been using Elixir for the past two years or so, and I’ve noticed that every single JSON library has it’s own benchmarks, that all say that they are the best. I wanted to know which one was the fastest currently, so I decided to build my own benchmarks. I’d love inputs from everyone to see what people think!

To see the benchmarks, you can look here, under the outputs folder: Elixir Random Benchmarks

That repo also contains various benchmarks of Elixir core that I’ve used to find the fastest versions of different functionalities in the core Elixir language.

Most Liked

michalmuskala

michalmuskala

Jiffy is generally faster on single-threaded benchmarks, the difference is much less pronounced when you start running concurrent processes all doing JSON stuff - Jason being all Elixir means it uses schedulers better - Jiffy being in C has a bit harder time dealing with that. At least those were my findings when I was benchmarking it around the initial release.

Also - the fact that it is in C is problematic, especially on Windows. It raises the entry bar considerably if you need to compile C on Windows, which many beginners use. I can only speculate this could be a reason for Phoenix sticking with a pure-elixir library as a default.

Additionally, especially with encoding, Jason can be much faster in practice. Jiffy can only deal with simple data types - strings, integers, maps - you need to encode everything down to that before handing it off to jiffy - especially annoying with things like DateTime or similar. This usually means two-step encoding - from your application to simple terms and from that to JSON. Because Jason is based on protocols, in can do all of that in a single, extensible pass. Furthermore, with deriving and the macros in Jason.Helpers module, you can do some additional work at compile-time to speed up the encoding even further. I wouldn’t be surprised if all of that noticeably outperforms jiffy.

I also have some ideas that would make it possible to do single-pass decoding into more complex data structures - in a similar way this should give an edge to Jason over jiffy if you don’t need to go over your data twice. At the same time it’s hard to find an API that would be both fast and extensible. As a last point when it comes to performance, when initially writing Jason, I had plans to include some optional native code to speed up the hottest loops - but there wasn’t that much demand for it so far, so it’s waiting for better times :wink: .

michalmuskala

michalmuskala

Since serde_rustler is a NIF that doesn’t yield, it should be running on dirty schedulers - after a quick glance at the code, I think it doesn’t do that right now. In production environment, NIFs that don’t yield can wreck havoc on the runtime. BEAM relies on the fact that scheduler threads checkpoint frequently with some VM services. Arranging NIF code so that in can yield properly usually has noticeable runtime cost - that’s probably the main source of speed-up over jiffy for your code. Running on dirty schedulers, on the other hand, has the context switch overhead, but more importantly has the caveats of running on a thread pool - if you exhaust the thread pool, the operations become blocking in practice.

What I meant about the parallel option is that the variations in the results are going to be much higher - that’s expected and has various sources - the VM overhead, other processes in the OS, etc.

keathley

keathley

We do what @michalmuskala suggests. We’ve benchmarked Jason and Jiffy with each of our individual services and we go with the one that makes the most sense.

But JSON is a garbage data format and we’re actively exploring alternative formats for our service communication layer.

Last Post!

OvermindDL1

OvermindDL1

If you want to compare json with a couple things, flatbuffer’s benchmark includes those (the super fast rapidjson, which is a lot faster than anything in elixir) and so forth:

Flatbuffers, due to knowing the possible schema, is significantly faster than rapidjson in every single way in binary mode, and when encoding to/from flatbuffer’s json it is still significantly faster than rapidjson, just not as fast (still a lot faster than just 3x-4x speedup that msgpack supposedly gives), and flatbuffer’s binary format is significantly smaller message size too.

Where Next?

Popular in Discussions Top

New
scouten
I’m looking for a host for the server part of a small (personal) side project that I’m working on. It’s currently written in Node.js and ...
New
AstonJ
Are there any Elixir or Erlang libraries that help with this? I’ve been thinking how streaming services like twitch have exploded recentl...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 40082 209
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
paulanthonywilson
I like Umbrella projects and pretty much always use them for personal Elixir stuff, especially Nerves things. But I don’t think this is ...
New

Other popular topics Top

baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49134 226
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

We're in Beta

About us Mission Statement