coby

coby

JSON Library Benchmarks

Hi Everyone!

First time writer, long time reader here.

I’ve been using Elixir for the past two years or so, and I’ve noticed that every single JSON library has it’s own benchmarks, that all say that they are the best. I wanted to know which one was the fastest currently, so I decided to build my own benchmarks. I’d love inputs from everyone to see what people think!

To see the benchmarks, you can look here, under the outputs folder: Elixir Random Benchmarks

That repo also contains various benchmarks of Elixir core that I’ve used to find the fastest versions of different functionalities in the core Elixir language.

First Post!

dimitarvp

dimitarvp

As far as I’m aware, using Jason is the conventional wisdom in the Elixir circles for a while now.

Most Liked

michalmuskala

michalmuskala

Jiffy is generally faster on single-threaded benchmarks, the difference is much less pronounced when you start running concurrent processes all doing JSON stuff - Jason being all Elixir means it uses schedulers better - Jiffy being in C has a bit harder time dealing with that. At least those were my findings when I was benchmarking it around the initial release.

Also - the fact that it is in C is problematic, especially on Windows. It raises the entry bar considerably if you need to compile C on Windows, which many beginners use. I can only speculate this could be a reason for Phoenix sticking with a pure-elixir library as a default.

Additionally, especially with encoding, Jason can be much faster in practice. Jiffy can only deal with simple data types - strings, integers, maps - you need to encode everything down to that before handing it off to jiffy - especially annoying with things like DateTime or similar. This usually means two-step encoding - from your application to simple terms and from that to JSON. Because Jason is based on protocols, in can do all of that in a single, extensible pass. Furthermore, with deriving and the macros in Jason.Helpers module, you can do some additional work at compile-time to speed up the encoding even further. I wouldn’t be surprised if all of that noticeably outperforms jiffy.

I also have some ideas that would make it possible to do single-pass decoding into more complex data structures - in a similar way this should give an edge to Jason over jiffy if you don’t need to go over your data twice. At the same time it’s hard to find an API that would be both fast and extensible. As a last point when it comes to performance, when initially writing Jason, I had plans to include some optional native code to speed up the hottest loops - but there wasn’t that much demand for it so far, so it’s waiting for better times :wink: .

michalmuskala

michalmuskala

Since serde_rustler is a NIF that doesn’t yield, it should be running on dirty schedulers - after a quick glance at the code, I think it doesn’t do that right now. In production environment, NIFs that don’t yield can wreck havoc on the runtime. BEAM relies on the fact that scheduler threads checkpoint frequently with some VM services. Arranging NIF code so that in can yield properly usually has noticeable runtime cost - that’s probably the main source of speed-up over jiffy for your code. Running on dirty schedulers, on the other hand, has the context switch overhead, but more importantly has the caveats of running on a thread pool - if you exhaust the thread pool, the operations become blocking in practice.

What I meant about the parallel option is that the variations in the results are going to be much higher - that’s expected and has various sources - the VM overhead, other processes in the OS, etc.

keathley

keathley

We do what @michalmuskala suggests. We’ve benchmarked Jason and Jiffy with each of our individual services and we go with the one that makes the most sense.

But JSON is a garbage data format and we’re actively exploring alternative formats for our service communication layer.

Last Post!

OvermindDL1

OvermindDL1

If you want to compare json with a couple things, flatbuffer’s benchmark includes those (the super fast rapidjson, which is a lot faster than anything in elixir) and so forth:

Flatbuffers, due to knowing the possible schema, is significantly faster than rapidjson in every single way in binary mode, and when encoding to/from flatbuffer’s json it is still significantly faster than rapidjson, just not as fast (still a lot faster than just 3x-4x speedup that msgpack supposedly gives), and flatbuffer’s binary format is significantly smaller message size too.

Where Next?

Popular in Discussions Top

PragTob
Hey everyone, this has been on my mind for some time and I’d love your input on it! TLDR: I feel like maps are superioer for storing and...
New
sergio
There’s a new TIOBE index report that came out that shows Elixir is still not in the top 50 used languages. It also goes on to call Elix...
New
tmbb
This is a post to discuss the new Phoenix LiveView functionality. From Chris’s talk, it appears that they generate all HTML on the serve...
342 18595 126
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 40082 209
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 14362 124
New
thojanssens1
It would be nice to be able to define a redirect from one route to another from the router.ex file. E.g.: redirect "/", UserController, ...
New
cvkmohan
The upcoming Phoenix 1.6 release looks very interesting. Became a habit to watch the commits - and - what they are bringing in. phx.gen...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement