ServerSentEvent library for parsing and consuming as a Client

Crowdhailer · December 19, 2018, 3:55pm

0.4.7 Make parsing 2_000x faster.

Just that.

See issue for benchmark and discussions.

github.com/CrowdHailer/server_sent_event.ex

More performant newline splitting

CrowdHailer:master ← nietaki:newline-splitting

opened 08:17PM - 18 Dec 18 UTC

nietaki

+143 -6

The issue popped up while using `server_sent_event` that the service using it sp…ent a lot of the time running regular expression code. It seems like it was caused by splitting binaries by newlines using regular expressions. The root cause is that even if you specify `parts: 2` in the `split` function options, if you're splitting by a regular expression, the whole string is going to be processed instead of just the part you need. See relevant code: - https://github.com/elixir-lang/elixir/blob/v1.6.4/lib/elixir/lib/string.ex#L383-L385 - https://github.com/elixir-lang/elixir/blob/v1.6.4/lib/elixir/lib/regex.ex#L472-L477 This approach should be much faster, as per the benchmark results: ``` Name ips average deviation median 99th % with_binary_split_compiled 10114.35 K 0.0989 μs ±177.83% 0.0900 μs 0.160 μs with_binary_split 806.20 K 1.24 μs ±114.65% 1.20 μs 2.10 μs with_lazy_compiled_pattern 717.57 K 1.39 μs ±88.91% 1.30 μs 2.40 μs with_elixir_string_split 669.83 K 1.49 μs ±76.74% 1.40 μs 2.60 μs with_erlang_regex 0.43 K 2350.80 μs ±3.82% 2342 μs 2697.22 μs with_regex 0.37 K 2698.95 μs ±5.03% 2651 μs 3193.58 μs Comparison: with_binary_split_compiled 10114.35 K with_binary_split 806.20 K - 12.55x slower with_lazy_compiled_pattern 717.57 K - 14.10x slower with_elixir_string_split 669.83 K - 15.10x slower with_erlang_regex 0.43 K - 23776.81x slower with_regex 0.37 K - 27298.11x slowe ``` ## Notes - More performance gains are possible by moving from `String.split` to `:binary.split`, it could introduce some confusion to an otherwise consistent Elixir'y code - Further performance gains are possible using compiling the patterns, (see the benchmark), but because it's not possible to store the compiled pattern as a module attribute, it would lead to much uglier code, at least using the approaches I thought about so far