Performance discrepancies with using QLC queries written in Erlang and Elixir

Hi folks,

In a library I’m working on we allow the user to implement “cache adapters” that implement our cache behaviour and the library can work with it.

I’ve recently introduced Erlang’s QLC module and I find it very intuitive to work with: we can now essentially ask users to implement a QLC table and all other read functions can then be defined by our library, with no extra work for the user. It also supports joins, which comes in very handy

The reason I’m posting is that I was benchmarking this in comparison to our previous implementation and realized that when the QLC query is written in Elixir using :qlc.string_to_handle/3 performance seems to be worse than when it is written natively in Erlang using the parse transform. A small excerpt (the native means the QLCs are written directly in Erlang):

Name                                     ips        average  deviation         median         99th %
native, naive                      388190.23     0.00258 ms   ±679.57%     0.00188 ms     0.00939 ms
naive                               25145.54      0.0398 ms    ±20.92%      0.0373 ms      0.0626 ms
native, nested                         23.21       43.09 ms    ±10.36%       41.71 ms       65.80 ms
native, naive, traversed               23.06       43.36 ms     ±9.46%       42.61 ms       59.01 ms
naive, traversed                        1.55      644.14 ms     ±2.32%      639.93 ms      665.63 ms
nested                                  0.36     2776.43 ms     ±0.00%     2776.43 ms     2776.43 ms

In terms of memory usage, the queries written in Erlang also seem to perform better (table size is 100_000 elements):

Name                              Memory usage
native, naive                       0.00218 MB
naive                                0.0116 MB - 5.31x memory usage +0.00941 MB
native, nested                        12.75 MB - 5844.20x memory usage +12.75 MB
native, naive, traversed              12.75 MB - 5845.10x memory usage +12.75 MB
naive, traversed                     346.34 MB - 158723.67x memory usage +346.33 MB
nested                              1064.87 MB - 488021.91x memory usage +1064.86 MB

The nested query is a bit special because it sticks two query handles together. I imagine that perhaps in the Erlang module there is some optimization going on that I can’t do from Elixir with string_to_handle.

However, for the naive query, the speed difference is not very clear to me. Did I maybe make a mistake in benchmarking? The query setup in Elixir:

qh = :ets.table(tab)
qh0 = :qlc.string_to_handle('[{Id, Id, Value} || {Id, Value} <- Handle, Id =:= RequestedId].', [], Handle: qh, RequestedId: 500_000)

And in Erlang (qh is passed in by the benchmarking script):

qh0(Tab, RequestedId) ->
    qlc:q([{Id, Id, Value} || {Id, Value} <- Tab, Id =:= RequestedId]).

If I print the query information using :qlc.info I see the same for both Erlang and Elixir. What could explain this difference?

The full benchmarking script I used can be found here: Benchmarking of Erlang's QLC module in various ways · GitHub