Jaxon: A very fast and flexible JSON parser with streaming support

Hello ,

Why another JSON parser? While working on my side project Sqlify.io, which is written in Elixir, I was in need of a JSON parser capable of parsing gigabytes worth of JSON documents without holding them all in memory and crash the VM, at the time I started using JSX but parsing big files was horribly slow, so inspired by Jiffy I started making my own parser, and started working on Jaxon.

Here’s a quick overview on how Jaxon works:

Jaxon’s default parser, which is implemented as a NIF (other implementations can be provided), takes a JSON document binary and returns a list of events like so:

iex(1)> Jaxon.Parser.parse(~s({"key":true}))
[:start_object, {:string, "key"}, :colon, {:boolean, true}, :end_object]

Then the decoder takes a list of events and evaluates them to an Elixir term:

iex(1)> Jaxon.Decoder.events_to_term([:start_object, {:string, "key"}, :colon, {:boolean, true}
, :end_object])
{:ok, %{"key" => true}}

There’s much more to it but as you can imagine, doing it this way means that both parsing and decoding can happen in a streaming fashion like so:

"large_file.json"
|> File.stream!()
|> Jaxon.Stream.query(Jaxon.Path.decode!("$.users.id"))
|> Enum.to_list()

Interested? here are some links to check it out:

Github: https://github.com/boudra/jaxon
Hex.pm: https://hex.pm/packages/jaxon

Thanks