hst337

hst337

Benchmarking dot in map.field

The benchmark

Mix.install [:benchee]

defmodule X do

  def dot(map) do
    [
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
    ]
  end

  def annotated(%{} = map) do
    [
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
      map.x + map.y,
      map.y + map.x,
    ]
  end

  def match(%{x: x, y: y}) do
    [
      x + y,
      y + x,
      x + y,
      y + x,
      x + y,
      y + x,
    ]
  end

  defmacrop dot(map, key) do
    quote do
      :erlang.map_get(unquote(key), unquote(map))
    end
  end

  def safe_dot(map) do
    [
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
      dot(map, :x) + dot(map, :y),
      dot(map, :y) + dot(map, :x),
    ]
  end

  defmacrop match(map, field) do
    quote do
      %{unquote(field) => value} = unquote(map)
      value
    end
  end

  def match_every_time(map) do
    [
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
      match(map, :x) + match(map, :y),
      match(map, :y) + match(map, :x),
    ]
  end

end

Benchee.run(%{
  "dot" => fn -> X.dot(%{x: 1, y: 1}) end,
  "annotated" => fn -> X.annotated(%{x: 1, y: 1}) end,
  "erlang.map_get" => fn -> X.safe_dot(%{x: 1, y: 1}) end,
  "match" => fn -> X.match(%{x: 1, y: 1}) end,
  "match_every_time" => fn -> X.match_every_time(%{x: 1, y: 1}) end,
}, warmup: 1, time: 2)

And the result is:

Name                       ips        average  deviation         median         99th %
match                   7.46 M      133.99 ns ±22658.59%          90 ns         184 ns
match_every_time        7.33 M      136.44 ns ±23751.72%          93 ns         181 ns
annotated               7.22 M      138.59 ns ±23522.99%          92 ns         196 ns
erlang.map_get          6.67 M      149.82 ns ±21303.83%         105 ns         212 ns
dot                     4.24 M      235.62 ns ±16830.96%         137 ns         266 ns

Comparison:
match                   7.46 M
match_every_time        7.33 M - 1.02x slower +2.45 ns
annotated               7.22 M - 1.03x slower +4.60 ns
erlang.map_get          6.67 M - 1.12x slower +15.83 ns
dot                     4.24 M - 1.76x slower +101.63 ns

Explanation

First of all, construction like map.field get’s compiled into this by elixir compiler. This is a feature of Elixir runtime and it is caused by feature of calling functions without (). However, code map.field() gets compiled to the same expression.

case map do
  module when is_atom(module) -> apply(module, :field, [])
  %{field: value} -> value
  _ -> raise BadMapError
end
  • match generates the fastest core_erlang and beam for this problem. Rule of thumb: pattern matching is always the fastest way to access the data in collections in both Erlang and Elixir.
  • annotated generates almost the same core_erlang as dot version, but compiler sees that the map variable in the function body will always be a map (otherwise it wouldn’t pass the matching in args) and it optimizes the case generated by dot to just a pattern matching. So it generates the same BEAM as match version
  • erlang.map_get is a very unpopular way to access the key of a map, and it is the third way to do so, added in one of the latest OTP version. It is actually a separate BIF and it is slower because it performs a map type check on every call
  • dot is the slowest one, because it performs a type check and a jump on every key access

Conclusions

  1. Annotate your maps to have better performance of ., or at least try to express as much code in pattern matching as possible
  2. Rewrite nested access like map.submap.subsubmap.field into pattern-matching, or at least use Pathex.
  3. Use Tria optimizing compiler which performs analysis to simplify the case generated by ., thus extracting the best performance for the . expression.

First Post!

lud

lud

You are extracting the values only once here.

Most Liked

mat-hek

mat-hek

Membrane Core Team

Thanks for the benchmark and for your engagement in making Elixir faster @hst337. Keep it up :raised_hands:

I bumped into the considered issue a while ago, when doing flame graph analysis which showed that get_in & friends are taking a lot of time. Replacing them with macro-generated pattern matching reduced the CPU usage by ~10% in our use case. A few similar hot path optimisations gave another 10%.

In some cases these optimisations give real speed up and having them automated would be invaluable.

hst337

hst337

Nah, it is not going to be implemented in the future, because it is just too much work for Elixir team. I’ve implemented it, but my implementation has insignificant limitations like slightly slower runtime recompilation. My ideas won’t become a part of the language any time soon. Core team of Elixir is interested in static typing, while core team of Erlang is interested in their own customers, who are in this 0.1% of users, who actually use hot-reloading in production

There was no reason in introducing this kind of syntax in the first place. Calls without parentheses could be implemented without this kind of performance drawback, like constructs where the left argument of . is not a compile-time atom and there are no parentheses or args, could always be treated as :erlang.map_get.

So this is a language design issue, and, to be honest, it is not very much of an important issue. I wrote a compiler which is able to overcome it, and simple annotation restores the performance back to 100%. Other languages have UBs, untrackable exceptions and forkbombs in 7 chars, so creators of Elixir did a good job there.

No, Moore’s law stoped working a while ago, and maximum number of cores in one CPU, and maximum number of CPU’s in one motherboard have their limits. We either optimize our code, or hope for the best in terms of quantum computing

hst337

hst337

I am not, and I’ve been in opposition to academia for my whole life.

I’d like to discuss it, since I am unaware about features my compiler is breaking. One of my goals from the beginning was 100% compatibility with existing behaviour of runtime, compiler and how people expect language to behave. Even these hard problems as runtime recompilation are under my aim in the moment, and they’re supported, but not completely since it is work in progress. If I am missing something, we can discuss it here, or in a compiler’s thread.

While it can be without this lag. And this is a problem. And it can be fixed. And it is fixed in my compiler. Performance optimizations in Elixir is one of thing I’ve decided to work on, this is not a holywar, and I am not attacking anybody with my posts, I am just sharing my experience about problems and solutions to them.

If you want to argue for the sake of it, without any particular reason, then you can, but please, do not involve me into this.

Last Post!

D4no0

D4no0

This is what I kind of understood from the posts on this forum, but once again I never went into much detail about the specifics since this is not interesting for me, I think there was something about runtime modules?

This is great that you want to go the 100% compatible way, and it will be amazing if we got at least a part of performance boost that these privileged languages get nowadays.

This is not for the sake of it, the idea is that you shouldn’t be stopped from using a feature, because currently is inefficient.

This is as you said a compiler implementation problem, and once fixed, everybody will be happy. My thinking is that a developer using a high-level language should not care about internals like these, because they are prone to change (and 100% he will not keep track of these changes and then mislead others on what to use and when) and because it fills the head with unnecessary information, instead of understanding how to write maintainable and readable code.

Where Next?

Popular in Discussions Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 55125 245
New
AlexMcConnell
The reason that Rails is as popular as it is is because it’s very easy for relatively inexperienced developers to get a lot of work done....
588 20142 166
New
joeerl
I’m playing with Elixir - It’s fun. I think @rvirding does give Elixir courses these days. Re: files and database - when I given Erlang ...
New
New
nburkley
AWS re:Invent is on at the moment with some interesting announcements. One new feature in particular is the Lambda Runtime API for AWS La...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
fireproofsocks
I’ve been working on an Elixir project that has required a lot of scripting. I usually reach for Elixir because I like it more (and in th...
New

Other popular topics Top

vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 49266 226
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New

We're in Beta

About us Mission Statement