JSON Library Benchmarks

Jiffy is generally faster on single-threaded benchmarks, the difference is much less pronounced when you start running concurrent processes all doing JSON stuff - Jason being all Elixir means it uses schedulers better - Jiffy being in C has a bit harder time dealing with that. At least those were my findings when I was benchmarking it around the initial release.

Also - the fact that it is in C is problematic, especially on Windows. It raises the entry bar considerably if you need to compile C on Windows, which many beginners use. I can only speculate this could be a reason for Phoenix sticking with a pure-elixir library as a default.

Additionally, especially with encoding, Jason can be much faster in practice. Jiffy can only deal with simple data types - strings, integers, maps - you need to encode everything down to that before handing it off to jiffy - especially annoying with things like DateTime or similar. This usually means two-step encoding - from your application to simple terms and from that to JSON. Because Jason is based on protocols, in can do all of that in a single, extensible pass. Furthermore, with deriving and the macros in Jason.Helpers module, you can do some additional work at compile-time to speed up the encoding even further. I wouldn’t be surprised if all of that noticeably outperforms jiffy.

I also have some ideas that would make it possible to do single-pass decoding into more complex data structures - in a similar way this should give an edge to Jason over jiffy if you don’t need to go over your data twice. At the same time it’s hard to find an API that would be both fast and extensible. As a last point when it comes to performance, when initially writing Jason, I had plans to include some optional native code to speed up the hottest loops - but there wasn’t that much demand for it so far, so it’s waiting for better times .

Since serde_rustler is a NIF that doesn’t yield, it should be running on dirty schedulers - after a quick glance at the code, I think it doesn’t do that right now. In production environment, NIFs that don’t yield can wreck havoc on the runtime. BEAM relies on the fact that scheduler threads checkpoint frequently with some VM services. Arranging NIF code so that in can yield properly usually has noticeable runtime cost - that’s probably the main source of speed-up over jiffy for your code. Running on dirty schedulers, on the other hand, has the context switch overhead, but more importantly has the caveats of running on a thread pool - if you exhaust the thread pool, the operations become blocking in practice.

What I meant about the parallel option is that the variations in the results are going to be much higher - that’s expected and has various sources - the VM overhead, other processes in the OS, etc.

We do what @michalmuskala suggests. We’ve benchmarked Jason and Jiffy with each of our individual services and we go with the one that makes the most sense.

But JSON is a garbage data format and we’re actively exploring alternative formats for our service communication layer.