After running mix clean and then say mix test I see all my files and modules compile one by one. During the process I have plenty of CPU underutilized and I ask myself why the compiler can’t utilize more cores on my machine?
PS: I’m not a compiler geek so I’m not looking for a deep answer, just kind of curious.
Is it possible you are conflating compiler parallelism and test parallelism? Do you see a difference is you just do mix compile —force? At the point you should see many cores being used unless, as @OvermindDL1 says, your dependency graph prevents it.
Yes, Elixir compiles files in parallel for a long time! It is one of the first articles published on the official website (for those who are interested in the technical aspects).
How large is your project? It may be that your project is small enough (and compilation is fast enough) that we actually don’t have enough time to warm up all of your CPUs. For example, when compiling hexpm, my quad-core maxes at 91% core usage and it still finishes in 5.5 seconds. When you say you see all your files and modules one by one, how are you visualizing that?
Only thing --force does is recompile everything, even unchanged files.
Elixir also compiles a LOT faster than most optimized compiled languages like swift/objc(/c/c++/rust/etc…) so it could just be running fast enough to not catch it.
Now the mix build system will build ‘dependencies’ one at a time, rather than in parallel as the graph allows, that could probably be fixed, but its generally not an issue either as those don’t recompile often, but ‘inside’ each dependency is still compiled in parallel too, which makes parallel dependency building even less important.
Yeah front-end assets is obviously not elixir, so that is all based on the node system, I.E. inherently very single-core.
On this, things like Compiling 12 files (.ex) means it is compiling 12 files from the dependency scribe in parallel (as their dependency graph allows).