Yes, maybe I can.
While I don’t have control over the CI machine, maybe I could change the job script so that pulls in the elixir tarball, builds it, and then let it use that one instead of the one we have deployed. Maybe. 
Oliver
1
Oliver
Well, we eventually upgraded to 1.14.4 (OTP25), and I remember expecting it to improve because dependency analysis is said to be improved, but the load average stays at 4 for 16 virtual cores.
Ironically, I found this thread when trying to analyze the same problem four years later. 
I just didn’t check back then when we upgraded, but I see load averages remain where they were, I guess.
2
Oliver
Just to give a bit of an impression about the code base:
- 700+ files
- 450K lines of elixir code
- Biggest file, 302K lines (it’s generated) - about 380K in total in generated files.
- Most files range between 15-150 lines, though (of the remaining 70K lines)
So I guess I’ll look for a way to split the really large files, even though they would only occupy one core each, and see what happens. 
0
Oliver
Had some fun with the compiler.
I had a map of maps - it was (pretty-printed) 100K lines long. So I wanted to “optimize it.”
Instead of this:
%{ "A_KEY" => %{ group1_key1: "blurb",
....},
"ANOTHER_KEY" => ...
I decided to do this:
%{ "A_KEY" => AnotherModuleInAnotherFile.A_KEY.group,
"ANOTHER_KEY" => AnotherModuleInAnotherFile.ANOTHER_KEY.group
,,,}
# and in a separate file
defmodule AnotherModuleInAnotherFile.A_KEY do
def group do
%{ group1_key1: "blurb",
....}
end
end
The map had basically 155 sub-maps which I put into separate files.
My expectation was that this would be a way to split this huge map sensibly. The sub-maps were completey constant so I expected the implementations of the group/1 functions to simply return a pointer or reference to term storage and this to become super-cheap in terms of compilation. (And to be done in parallel.)
Instead I learned of a new compiler-error:
The supposedly complex instruction was described as: {call_ext,0,{extfunc,'Elixir.MyModule.MY_KEY',group,0}}
Also, compilation expanded from roughly 90s (for a total of 300K lines) into more than ten minutes when it got killed by a supervision process.
So much for my assumptions of how to make code compile in parallel. 
So I kept the huge maps, but split the file in three parts, and that’s what I can sensibly do here, I guess.
Alternatively I could instead put all the 155 entries into :persistent_term instead with 155 generated put/2 calls and use that. The map literal is just a module attribute used by a function for lookup. So, it could also simply refer to :persistent_term.get/1 without changing the interface.
I guess in that case the compiler would not be forced to resolve it the same way. (I assume the compiler simply tried to resolve the map literal just the same as before, but the ext functions made its work more complex…)
0
benwilson512
Author of Craft GraphQL APIs in Elixir with Absinthe
Ever since persistent term there isn’t really any need to bake constants into macro generated code literals anymore. It’s way faster to write into, and you get the same (and sometimes better) performance for reads. And it’s way easier to use.
1
DianaOlympos
Something to note. Average core load of 4 does not mean only 4 cores are used
It could be that 16 are used but they spend most of their time on IO or idling or blocked by a lock somewhere. This is a not too rare thing to happen.
CPU load is a pretty flawed metric these days to evaluate how much you saturate the number of cores.
See CPU Utilization is Wrong and Linux Load Averages: Solving the Mystery as starting points
2
nicnilov
I came here looking for an answer after observing a simple livebook example focused on PartitionSupervisor which is supposed to show the best use of all cores. However in my case ([smp:12:12]) it was only utilizing 4 cores.
Note that the example for some reason sets max_concurrency: 2000 which I don’t have an explanation for. I was using it with max_concurrency: System.schedulers_online() instead. Going with much larger value expectedly reduces the throughput.
With this setup I was getting around 160K events per second, however my OS monitor was only showing 4 cores intermittently utilized. Notably, setting the max_concurrency: 4 or even 3 did not change the throughput at all which confirms the setup is not CPU-bound.
Following the previous comments it might be reasonable to assume the message copying between the processes and the associated memory IO is the culprit. I haven’t investigated deeper myself, but maybe someone would have an explanation to this behavior now that a reproducible code is available.
It seems to me even with a high memory IO due to message passing, in the presence of a scheduler per core the cpu utilization should be better. And then, what is the benefit of PartitionSupervisor if CPU is not the bottleneck?
0