Help with structs benchmarks

It’s been some time, but the benchmark we had wasn’t straightforward to run, so I made another one: membrane_benchmarks/core_beamchmark at master · membraneframework-labs/membrane_benchmarks · GitHub

Hope it’s useful

Sorry, it’s a bit later…

I realized that the core what we’re doing is working with a tree which is inspired by this GitHub - japplegame/avl_tree: Pure Elixir AVL tree implementation · GitHub

But that one above is using tuples, so I changed it to use structs. Here’s the fork: GitHub - stefanluptak/avl_tree at use-structs-instead-of-tuples · GitHub

There’s even a benchmark script already, so you can just run mix run bench/run.exs Just do not forget to use that particular branch on my fork.

I hope it will be helpful somehow.

Sorry for the delay, took a bit longer to get the schemas in order than I expected.

Here is a benchark that you can run with mix run bench.exs at the root of the project. It will parse a large XML in the manner I mentioned.

Let me know if you can’t access it for whatever reason.

Adz

Nice, yea no problem it is trivial to turn the Map.fetch! into a map.field

I will get something together today for you.

Yes, I would love to use this benchmark! However, note that we won’t be able to optimize Map.fetch but we should be able to optimize pattern matching:

case map do
  %{^key => value} -> ...
  %{} -> ...
end

Or map.field.

Oh!

Here’s a thing that might help. I’ve been (experimenting) with XML parsing. One iteration I did was creating a Saxy handler that parses the XML into a struct, something like the following,

Given this xml:

<ParentNode price="1">
  <MyNode>Stuff</MyNode>
  <AnotherNode>
    <Child>Stuff</Child>
  </AnotherNode>
</ParentNode>

The Saxy handler returns:

 %XMLNode{
   attributes: [%XMLAttr{name: "price", value: "1"}],
   content: [
     %XMLNode{attributes: [], content: ["Stuff"], name: "MyNode"},
     %XMLNode{
       attributes: [],
       content: [%XMLNode{attributes: [], content: ["Stuff"], name: "Child"}],
       name: "AnotherNode"
     }
   ],
   name: "ParentNode"
 }

THEN we have a library called data_schema that queries into that representation to create a different struct from it. It does that by reducing through a schema and querying into the result from Saxy, so if you have large XML you’ll be doing lots and lots of Map.gets into the result of the Saxy handler to get the values for your new struct.

I have created a branch on the data_schema repo called xpath_experiment that has the Saxy handler and a data accessor that you can use to parse XML.
See this example from the tests:

Doing this will result in a lot of Map.fetch into a struct - 1 per node and attr.

If you think this will be useful to you I can get together a large XML (~15mb) example to use, along with schemas for that.

OK, I will do my best. But since it’s Friday already, I will send you that on Monday.

@stefanluptak that would be perfect! An Elixir script with Mix.install at the top that installs the required depsand then proceeds to aggregate would be perfect!

@mat-hek if that can be isolated, that would be perfect too! Especially knowing the code was a bottleneck in the past.

I think rebuilding Commanded’s aggregate from a lot of events (when there’s no snapshot) could be a good example of that.
We’re doing that in our company too. I can provide you with some test code and data or maybe even a Benchee script to run, if you want.

In Membrane, we use nested structs and (usually small) maps for element state, which is basically a GenServer state. There are rather no heavy loops over it, but we figured out that access to structs impacts performance significantly. So we replaced Elixir’s Access by macros that construct pattern matches in the compile time. That turned out to provide significant speedup in some of our benchmarks. Not sure if that’s what you’re looking for, but I guess speeding up structs/maps themselves could give Membrane another boost.