This is definitely unexpected. I’m getting these results:

$ elixir -e "Mix.install [:benchee] ; Benchee.run([])" 2>/dev/null | head -7
Operating System: macOS
CPU Information: Apple M2
Number of Available Cores: 8
Available memory: 24 GB
Elixir 1.18.1
Erlang 27.2
JIT enabled: true
$ which elixir ; which erl
/Users/wojtek/.elixir-install/installs/elixir/1.18.1-otp-27/bin/elixir
/Users/wojtek/.elixir-install/installs/otp/27.2/bin/erl
$ hyperfine --warmup 2 -S none -- "elixir -e 'System.halt(0)'" "elixir -e 'IO.puts(:ok)'"
Benchmark 1: elixir -e 'System.halt(0)'
  Time (mean ± σ):     127.5 ms ±   1.1 ms    [User: 127.0 ms, System: 129.1 ms]
  Range (min … max):   125.7 ms … 129.6 ms    23 runs

Benchmark 2: elixir -e 'IO.puts(:ok)'
  Time (mean ± σ):     129.0 ms ±   0.9 ms    [User: 129.0 ms, System: 133.1 ms]
  Range (min … max):   126.9 ms … 131.0 ms    23 runs

Summary
  elixir -e 'System.halt(0)' ran
    1.01 ± 0.01 times faster than elixir -e 'IO.puts(:ok)'

Could you double-check you are benchmarking with elixir/otp installed via install script? What OTP are you using?

Most of time is spent booting the VM so I’d focus on that:

$ hyperfine --warmup 2 -S none -- "erl -noshell -eval 'halt()'"
Benchmark 1: erl -noshell -eval 'halt()'
  Time (mean ± σ):      90.7 ms ±   1.2 ms    [User: 92.4 ms, System: 89.5 ms]
  Range (min … max):    88.5 ms …  92.5 ms    33 runs

Perhaps it’s the OTP builds that the installer is using, GitHub - erlef/otp_builds: Community-maintained pre-compiled Erlang/OTP for macOS · GitHub. What you can try is compiling OTP using kerl and re-running the benchmark. You can compile from scratch yourself too which is easy enough:

$ git clone https://github.com/erlang/otp --branch OTP-27.2 --depth 1
$ cd otp
$ ulimit -n 1024 && ERL_TOP=$PWD MAKEFLAGS=-j8 ./otp_build setup
$ ./bin/erl -noshell -eval 'halt()'
$ hyperfine --warmup 2 -S none -- "bin/erl -noshell -eval 'halt()'"

The ranges seem unusually large - benchmark 2 shows almost a 10x variation between the fastest and slowest run.

Is it possible your terminal is somehow running in Rosetta? That seems far less likely than it was in the early M1 days, but it’s the first thing I can think of that would produce that much inter-run variation (since it caches translated code).

One other random thing is what ulimits do you have set? I had to increase the laughable low default limit on my new M4 machine I set up recently.