Elixir is slow on my Macbook (M3 Pro/18GB)

Questions / Help
, ,
1

M3 Pro with 18 GB memory

Fresh install of Elixir 1.18.1 via the install.sh script: https://elixir-lang.org/install#install-scripts

My understanding is that these times should be well under a second and they are definitely not.

[I] ~/elixir-install > hyperfine --warmup 2 -S none --export-markdown results.md -- 'elixir -e \'System.halt(0)\'' 'elixir -e \'IO.puts("Hello world")\''

Benchmark 1: elixir -e 'System.halt(0)'
  Time (mean ± σ):      2.592 s ±  1.754 s    [User: 0.163 s, System: 0.186 s]
  Range (min … max):    0.494 s …  5.542 s    10 runs

Benchmark 2: elixir -e 'IO.puts("Hello world")'
  Time (mean ± σ):      7.328 s ±  5.937 s    [User: 0.264 s, System: 0.148 s]
  Range (min … max):    2.528 s … 23.440 s    10 runs

Summary
  elixir -e 'System.halt(0)' ran
    2.83 ± 2.98 times faster than elixir -e 'IO.puts("Hello world")'

Not sure what is causing this. Need help digging in.

This seems to be exactly what I am experiencing: Elixir and Mac OS - anyone noticed a drop in performance? — but does not look like a resolution was ever found.

1 Like
2

This is definitely unexpected. I’m getting these results:

$ elixir -e "Mix.install [:benchee] ; Benchee.run([])" 2>/dev/null | head -7
Operating System: macOS
CPU Information: Apple M2
Number of Available Cores: 8
Available memory: 24 GB
Elixir 1.18.1
Erlang 27.2
JIT enabled: true
$ which elixir ; which erl
/Users/wojtek/.elixir-install/installs/elixir/1.18.1-otp-27/bin/elixir
/Users/wojtek/.elixir-install/installs/otp/27.2/bin/erl
$ hyperfine --warmup 2 -S none -- "elixir -e 'System.halt(0)'" "elixir -e 'IO.puts(:ok)'"
Benchmark 1: elixir -e 'System.halt(0)'
  Time (mean ± σ):     127.5 ms ±   1.1 ms    [User: 127.0 ms, System: 129.1 ms]
  Range (min … max):   125.7 ms … 129.6 ms    23 runs

Benchmark 2: elixir -e 'IO.puts(:ok)'
  Time (mean ± σ):     129.0 ms ±   0.9 ms    [User: 129.0 ms, System: 133.1 ms]
  Range (min … max):   126.9 ms … 131.0 ms    23 runs

Summary
  elixir -e 'System.halt(0)' ran
    1.01 ± 0.01 times faster than elixir -e 'IO.puts(:ok)'

Could you double-check you are benchmarking with elixir/otp installed via install script? What OTP are you using?

Most of time is spent booting the VM so I’d focus on that:

$ hyperfine --warmup 2 -S none -- "erl -noshell -eval 'halt()'"
Benchmark 1: erl -noshell -eval 'halt()'
  Time (mean ± σ):      90.7 ms ±   1.2 ms    [User: 92.4 ms, System: 89.5 ms]
  Range (min … max):    88.5 ms …  92.5 ms    33 runs

Perhaps it’s the OTP builds that the installer is using, GitHub - erlef/otp_builds: Community-maintained pre-compiled Erlang/OTP for macOS. What you can try is compiling OTP using kerl and re-running the benchmark. You can compile from scratch yourself too which is easy enough:

$ git clone https://github.com/erlang/otp --branch OTP-27.2 --depth 1
$ cd otp
$ ulimit -n 1024 && ERL_TOP=$PWD MAKEFLAGS=-j8 ./otp_build setup
$ ./bin/erl -noshell -eval 'halt()'
$ hyperfine --warmup 2 -S none -- "bin/erl -noshell -eval 'halt()'"
2 Likes
3

More related to the “anyone noticed a drop in performance” link that you posted, and not really likely to be of any help, but a few years ago I noticed that elixir started up much more slowly than any of my other scripting languages. I had at one time I had a file showing the startup times of all of them but I can’t find it so I must have deleted it.

In any case, the problem went away. I can’t say that it went away because of an update to the OS or an update to elixir, and in fact I don’t remember ever knowing why it went away, but it did.

1 Like
4

The ranges seem unusually large - benchmark 2 shows almost a 10x variation between the fastest and slowest run. :thinking:

Is it possible your terminal is somehow running in Rosetta? That seems far less likely than it was in the early M1 days, but it’s the first thing I can think of that would produce that much inter-run variation (since it caches translated code).

2 Likes
5

One other random thing is what ulimits do you have set? I had to increase the laughable low default limit on my new M4 machine I set up recently.

2 Likes
6

A colleague of mine, had performance issues, because the shell, he started all from, ran with Rosetta emulation. He switched from MPB with intell to one with M3 and did not do the full brew reinstall things.

Edit:
Sorry, coffee level is still low, so I did not see, that this idea already popped up

1 Like
7

I got the same machine:

CPU Information: Apple M3 Pro
Number of Available Cores: 12
Available memory: 18 GB
Elixir 1.18.1
Erlang 27.1.2
JIT enabled: true

Results:

hyperfine --warmup 2 -S none -- "elixir -e 'System.halt(0)'" "elixir -e 'IO.puts(:ok)'"

Benchmark 1: elixir -e 'System.halt(0)'
  Time (mean ± σ):     117.8 ms ±   2.8 ms    [User: 119.6 ms, System: 241.1 ms]
  Range (min … max):   115.2 ms … 125.3 ms    25 runs

Benchmark 2: elixir -e 'IO.puts(:ok)'
  Time (mean ± σ):     119.3 ms ±   3.0 ms    [User: 121.4 ms, System: 251.3 ms]
  Range (min … max):   115.5 ms … 125.5 ms    25 runs

Summary
  elixir -e 'System.halt(0)' ran
    1.01 ± 0.03 times faster than elixir -e 'IO.puts(:ok)'
1 Like