Does this mean Ruby San, is the fastests fasters ?

(btw shouldn’t this be done with benchee or so?)

Seems about right. Would be nice to have Jiffy in there as well just for comparison sake.

One important thing to consider when looking at those numbers: the Go and Elixir solutions are written in pure Go and pure Elixir respectively.

I don’t know what they’re using on the Ruby side but most efficient code in Ruby/Python/PHP/Perl is just directly calling C.

The Ruby parser is implemented in C, so it’s more like comparing C to Elixir. Obviously C is going to be faster We’re not doing that bad actually. It would be interesting to compare with Jason compiled with HiPE. In my benchmarks this makes it at least twice as fast.

The data is also quite different from what you’d face in a regular HTTP app - the JSON is pretty-printed, most JSON flying on the wire is not. This can have significant difference in the actual performance. Depending on what you want to learn from this, it can be important.

There’s exactly 0 chance that Ruby is not just calling some C lib for this
Still, for native Elixir this is reassuringly fast.

Don’t suppose you’d be up for tweaking the Techempower benchmarks for Elixir to use Jason with HiPE. It’s the basis for the results of so much of those tests I think it would go a long way with the report.

Let’s have some fun, here is how they compare to C++/rapidjson:

# Golang
$ time jq '.' data/10mb.json > /dev/null

real    0m0.607s
user    0m0.603s
sys     0m0.004s

$ time jq '.' data/citylots.json > /dev/null

real    0m14.348s
user    0m13.839s
sys     0m0.504s

# Elixir
## 10mb.json
Time taken [Jason]: 521.655ms
Time taken [Poison]: 1358.531ms
## citylots.json
Time taken [Jason]: 12224.44ms
Time taken [Poison]: 33350.239ms

# Ruby
## 10mb.json
$ time ruby app.rb

real    0m0.350s
user    0m0.250s
sys     0m0.020s
## citylots.json
$ time ruby app.rb

real    0m5.632s
user    0m5.393s
sys     0m0.236s

# C++
$ time ./rapidjson-testing < ../../benchmark-large-json-parsing/data/10mb.json 

real    0m0.035s
user    0m0.034s
sys     0m0.000s

$ time ./rapidjson-testing < ../../benchmark-large-json-parsing/data/citylots.json 

real    0m0.531s
user    0m0.503s
sys     0m0.028s

Admittedly this benchmark is flawed because jq is outputting to stdout and elixir/ruby/C++ are just kind of blackholing the data after it is parsed, so jq/go is artificially limited here. In addition the elixir version is actually instancing a tree to hold the whole structure, which is wasted work as well (unsure about ruby). The C++ version is fully parsing and performing callbacks for every parse (standard sax parsing).

I can PR the C++ one in it if you want, it only needs the normal C++ compiler and cmake installed, nothing else needed (not even rapidjson, it acquires it itself).

Just to make sure, here is the C++ version as both a sax parser, and as an elixir-style-structure-building document parser (yay eating memory):

$ time ./rapidjson-sax < ../../benchmark-large-json-parsing/data/10mb.json 

real    0m0.038s
user    0m0.034s
sys     0m0.004s

$ time ./rapidjson-sax < ../../benchmark-large-json-parsing/data/citylots.json 

real    0m0.529s
user    0m0.485s
sys     0m0.044s

$ time ./rapidjson-structure < ../../benchmark-large-json-parsing/data/10mb.json 

real    0m0.037s
user    0m0.036s
sys     0m0.000s

$ time ./rapidjson-structure < ../../benchmark-large-json-parsing/data/citylots.json 

real    0m0.533s
user    0m0.504s
sys     0m0.028s

Not much of a difference, honestly the C++ compiler is so good that it is probably being optimized out, hmm…

EDIT: And I added some code to print out some details about the structure to ensure it is compiled and parsed in full and it somehow got a few milliseconds faster… so yeah those are accurate, C++ is just fast as always…

Would love a PR

You are right about jq doing more work, I’ll use the standard json parser and make it match the ruby and elixir versions.

Not sure you mean by this:

In addition the elixir version is actually instancing a tree to hold the whole structure, which is wasted work as well (unsure about ruby).

Elixir would parse the data and have it in memory.

The C++ version seems insane, I’d love to add a Rust benchmark too. I know this is all crude benchmarking but it gives you a sense of comparison.

I didn’t use Benchee because the run times are fairly large. I’ll see if I can add benchee with small json loads.