pejrich
Compiling files with many function heads is very slow(OTP 26 issue?)
I have quite a few files in my project that have many function heads(100s to 1000s). I don’t think these can easily be replaced with another way of doing it. For example, transliterating Japanese into Latin characters. Right now I have compile time code read from this Unihan_Readings.txt file[Warning, 5mb text file], and generate functions that match the Japanese character at the beginning of the string. This can’t be replaced by String.split, because some matches are multi-character, but the variation in byte length is even wider, so binary pattern matching is the most obvious solution.
Anyway, compiling a file like this, is currently taking a LONG time, and it seems to have gotten worse under OTP 26, like a lot worse. It used to take maybe 30-40 seconds, now it takes 20-30 minutes(M1 MBP running Ventura 13.4. OTP 26.0, Elixir 1.15.3-otp-26). Is there any way to optimize this ahead of time for quicker compilation?
I’m currently running a compile with ERL_COMPILER_OPTIONS=time mix compile --force --profile time, so far(45 minutes), here’s some of the output:
beam_ssa_codegen : 0.715 s 114644.0 kB
beam_validator_strong : 0.195 s 114644.0 kB
beam_a : 0.018 s 115052.9 kB
beam_block : 0.025 s 120606.8 kB
beam_jump : 0.105 s 100114.5 kB
beam_clean : 0.002 s 100114.5 kB
beam_trim : 0.000 s 100114.5 kB
beam_flatten : 0.000 s 99683.7 kB
beam_z : 0.000 s 99668.4 kB
beam_validator_weak : 0.099 s 99668.4 kB
beam_asm : 0.191 s 95366.4 kB
beam_ssa_opt : 1876.602 s 277640.0 kB
%% Sub passes of beam_ssa_opt from slowest to fastest:
ssa_opt_alias : 1866.038 s 99 %
ssa_opt_live : 2.007 s 0 %
ssa_opt_type_start : 1.973 s 0 %
ssa_opt_dead : 1.113 s 0 %
ssa_opt_type_continue : 0.958 s 0 %
ssa_opt_bsm_shortcut : 0.621 s 0 %
ssa_opt_merge_blocks : 0.447 s 0 %
ssa_opt_cse : 0.374 s 0 %
Most Liked
josevalim
Most likely the same as this one: Undesired(?) slow down in ssa_opt_alias · Issue #7432 · erlang/otp · GitHub - TL;DR: they are working on a fix.
This may also help in future cases: How to debug Elixir/Erlang compiler performance - Dashbit Blog
BartOtten
Wrote 3 simple examples to benchmark compile times. They perform similar without the guards but…it seems the Case-variant is immune for adding guards, while the others start to slow down.
Of course the benchmark is not realistic. Kept it simple, no macro in macro.
test1.ex compiled in 3203.686 ms. → Modules: [Anon]
test2.ex compiled in 1394.32 ms → Modules: [Case]
test3.ex compiled in 3040.71 ms → Modules: [Heads]
ps. Yes, naming variables is hard 
Snippets:
Case
defmodule Case do
case_ast =
for i <- 1..10000 do
{:->, [], [[{i, i}], i]}
end
def foo(x, y) when is_integer(x) and is_integer(y) do
case {x, y} do
unquote(case_ast)
end
end
end
Heads
defmodule Heads do
for i <- 1..10000 do
def foo(z = unquote(i), a= unquote(i)) when is_integer(z) and is_integer(a), do: unquote(i)
end
end
Anon function
defmodule Anon do
fun = fn x, y ->
def foo(z= unquote(x), a = unquote(y)) when is_integer(z) and is_integer(a), do: unquote(y)
end
for i <- 1..10000 do
fun.(i, i)
end
end
josevalim
This would be a more apples to apples version:
defmodule Case do
case_ast =
for i <- 1..10000 do
hd(
quote do
{x = unquote(i), y = unquote(i)} when is_integer(x) and is_integer(y) -> y
end
)
end
def foo(x, y) do
case {x, y} do
unquote(case_ast)
end
end
end
Now they all have to go through the same amount of guards and the end result is roughly the same on my machine. I also could not spot any difference between public and private.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Enabling companies to succeed by building software people love.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Real-time error tracking, performance insights, and observability for devs.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"