hst337

hst337

Benchmarking dot in map.field

The benchmark

Mix.install [:benchee]

defmodule X do

  def dot(map) do
    [
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
    ]
  end

  def annotated(%{} = map) do
    [
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
    ]
  end

  def match(%{x: x, y: y}) do
    [
      x + y,
      y + x,
      x + y,
      y + x,
      x + y,
      y + x,
    ]
  end

  defmacrop dot(map, key) do
    quote do
      :erlang.map_get(unquote(key), unquote(map))
    end
  end

  def safe_dot(map) do
    [
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
    ]
  end

  defmacrop match(map, field) do
    quote do
      %{unquote(field) => value} = unquote(map)
      value
    end
  end

  def match_every_time(map) do
    [
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
    ]
  end

end

Benchee.run(%{
  "dot" => fn -> X.dot(%{x: 1, y: 1}) end,
  "annotated" => fn -> X.annotated(%{x: 1, y: 1}) end,
  "erlang.map_get" => fn -> X.safe_dot(%{x: 1, y: 1}) end,
  "match" => fn -> X.match(%{x: 1, y: 1}) end,
  "match_every_time" => fn -> X.match_every_time(%{x: 1, y: 1}) end,
}, warmup: 1, time: 2)

And the result is:

Name                       ips        average  deviation         median         99th %
match                   7.46 M      133.99 ns ±22658.59%          90 ns         184 ns
match_every_time        7.33 M      136.44 ns ±23751.72%          93 ns         181 ns
annotated               7.22 M      138.59 ns ±23522.99%          92 ns         196 ns
erlang.map_get          6.67 M      149.82 ns ±21303.83%         105 ns         212 ns
dot                     4.24 M      235.62 ns ±16830.96%         137 ns         266 ns

Comparison:
match                   7.46 M
match_every_time        7.33 M - 1.02x slower +2.45 ns
annotated               7.22 M - 1.03x slower +4.60 ns
erlang.map_get          6.67 M - 1.12x slower +15.83 ns
dot                     4.24 M - 1.76x slower +101.63 ns

Explanation

First of all, construction like map.field get’s compiled into this by elixir compiler. This is a feature of Elixir runtime and it is caused by feature of calling functions without (). However, code map.field() gets compiled to the same expression.

case map do
  module when is_atom(module) -> apply(module, :field, [])
  %{field: value} -> value
  _ -> raise BadMapError
end
  • match generates the fastest core_erlang and beam for this problem. Rule of thumb: pattern matching is always the fastest way to access the data in collections in both Erlang and Elixir.
  • annotated generates almost the same core_erlang as dot version, but compiler sees that the map variable in the function body will always be a map (otherwise it wouldn’t pass the matching in args) and it optimizes the case generated by dot to just a pattern matching. So it generates the same BEAM as match version
  • erlang.map_get is a very unpopular way to access the key of a map, and it is the third way to do so, added in one of the latest OTP version. It is actually a separate BIF and it is slower because it performs a map type check on every call
  • dot is the slowest one, because it performs a type check and a jump on every key access

Conclusions

  1. Annotate your maps to have better performance of ., or at least try to express as much code in pattern matching as possible
  2. Rewrite nested access like map.submap.subsubmap.field into pattern-matching, or at least use Pathex.
  3. Use Tria optimizing compiler which performs analysis to simplify the case generated by ., thus extracting the best performance for the . expression.

Most Liked

mat-hek

mat-hek

Membrane Core Team

Thanks for the benchmark and for your engagement in making Elixir faster @hst337. Keep it up :raised_hands:

I bumped into the considered issue a while ago, when doing flame graph analysis which showed that get_in & friends are taking a lot of time. Replacing them with macro-generated pattern matching reduced the CPU usage by ~10% in our use case. A few similar hot path optimisations gave another 10%.

In some cases these optimisations give real speed up and having them automated would be invaluable.

hst337

hst337

Nah, it is not going to be implemented in the future, because it is just too much work for Elixir team. I’ve implemented it, but my implementation has insignificant limitations like slightly slower runtime recompilation. My ideas won’t become a part of the language any time soon. Core team of Elixir is interested in static typing, while core team of Erlang is interested in their own customers, who are in this 0.1% of users, who actually use hot-reloading in production

There was no reason in introducing this kind of syntax in the first place. Calls without parentheses could be implemented without this kind of performance drawback, like constructs where the left argument of . is not a compile-time atom and there are no parentheses or args, could always be treated as :erlang.map_get.

So this is a language design issue, and, to be honest, it is not very much of an important issue. I wrote a compiler which is able to overcome it, and simple annotation restores the performance back to 100%. Other languages have UBs, untrackable exceptions and forkbombs in 7 chars, so creators of Elixir did a good job there.

No, Moore’s law stoped working a while ago, and maximum number of cores in one CPU, and maximum number of CPU’s in one motherboard have their limits. We either optimize our code, or hope for the best in terms of quantum computing

hst337

hst337

I am not, and I’ve been in opposition to academia for my whole life.

I’d like to discuss it, since I am unaware about features my compiler is breaking. One of my goals from the beginning was 100% compatibility with existing behaviour of runtime, compiler and how people expect language to behave. Even these hard problems as runtime recompilation are under my aim in the moment, and they’re supported, but not completely since it is work in progress. If I am missing something, we can discuss it here, or in a compiler’s thread.

While it can be without this lag. And this is a problem. And it can be fixed. And it is fixed in my compiler. Performance optimizations in Elixir is one of thing I’ve decided to work on, this is not a holywar, and I am not attacking anybody with my posts, I am just sharing my experience about problems and solutions to them.

If you want to argue for the sake of it, without any particular reason, then you can, but please, do not involve me into this.

Where Next?

Popular in Discussions Top

blackode
Elixir Upgrading is so Simple in Ubuntu and It worked for me Ubuntu 16.04 git clone https://github.com/elixir-lang/elixir.git cd elixir...
New
arcanemachine
https://nitter.net/josevalim/status/1744395345872683471 https://twitter.com/josevalim/status/1744395345872683471
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 13924 124
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
Crowdhailer
I’ve been hearing much about the new formatter and it’s something I have been keen to try. I find examples buy far the most illuminating...
248 19204 150
New
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
acrolink
How does the two languages compare when it comes to server side application development? Any experiences or ideas? Thank you.
New
scouten
I’m looking for a host for the server part of a small (personal) side project that I’m working on. It’s currently written in Node.js and ...
New
griffinbyatt
Sobelow Sobelow is a security-focused static analysis tool for the Phoenix framework. For security researchers, it is a useful tool for g...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

We're in Beta

About us Mission Statement