There are indeed scenarios when this is the case, however I would say that if you are not running alone on a machine, you do not want to have spinning enabled as you start to conflict with other services. I’ve been thinking about maybe changing the default to none or very_short, as that seems to be what most systems needs.
I can also see how changing +swct and +swt could be useful, as they adjust how eager the VM is to wake up more schedulers to help do work.
Nice, setting the default to be what is best for shared environments makes sense.
For +swct what are the cleaning tasks it is referring to? Garbage collection?
Would lowering +swt be best for busty loads? So that it takes longer to wake up a scheduler in cases it will just have to go back to sleep shortly after, but if the burst is more sustained it will still get woken up? Or could it be an improvement for general workloads in a shared environment?
Glad you brought them up, I was planning to finally look at benchmarking these but wanted to get a grasp on how to best structure the benchmarks to be most likely to show where they help.
Curious also about the different scheduler balancing options. Do you think using one or the other, +scl or +sub, could be useful in a shared environment?
Internal cleanup work by the schedulers. For instance, delayed de-allocation of remote memory blocks. otp/erts/emulator/internal_doc/DelayedDealloc.md at master · erlang/otp · GitHub
Tinkering with +swt would be the classic tradeoff of latency vs CPU. If you set it to very_high then you will use less CPU to do the work as the work tends to be co-located, so there is a lesser risk of lock contention and better cache usage etc etc. However, at the same time, the average time a job will wait in the run-queue before being allowed to run will go up, so you application will have higher latency.
I don’t consider them relevant at all. The default strategy is best for all scenarios. They exist in order to support the strange needs of some embedded systems run at Ericsson. +scl false is very similar to what you get when you run +swt very_low, so it could possibly be good when you want to optimize for latency.
How about +swtdio very_low?
Could this improve performance in shared environments as well?
Yes. Infact, in OTP-22, it’s default had been changed to short: http://blog.erlang.org/cd/docs/master/erts-10.3.2/doc/html/erl.html#+sbwtdio
We run multiple dockerized Elixir Microservices on every host.
For the sake of others seeking performance optimization in shared environments in the following I summarize the changes to the vm.args file I currently try in one of our Services:
+sbwt very_short
+swt very_low
+swtdcpu very_low
+swtdio very_low
+swct very_eager
With these ‘optimizations’ I hope to achieve a noticeable performance gain. Let’s see what happens.
I don’t care about resource consumption but I want my applications to be as fast as possible. So shouldn’t, for example, schedulers be woken eagerly consuming more CPU but getting the work done faster?
Or am I misinterpreting the documentation here?
Or, since these two options swt and swct concern cleanup schedulers which should only run when CPU is not so busy as not to disturb the running applications?
Thanks for the reply!
@garazdawi Can you please elaborate on my misconception?
Sadly, the time in service with the previously mentioned changes increases by 2ms (21ms now), roughly 2%. So the result of this little test agrees with your statement on my customized emulator flags:
I think you may have set these options the wrong way around.
Which part of my reasoning is faulty?
As I understand the documentation the sbwt* flags should be set to longer in order to keep idle schedulers alive for longer so new work will be accepted faster without the need of starting a new scheduler.
On the other hand the swt* flags should be set to a lower value as to wake up schedulers earlier when new work comes in that the current schedulers can’t handle.
If you try to compact the load as much as possible, the emulator will run using one scheduler. You may get a bit higher latency for work in that specific instance. However, you will also have less threads running at the same time which should make it easier for the OS to schedule all of your containers efficiently.
So by compacting load more aggressively you will sacrifice performance in a specific emulator for the greater good of all the other systems running on the same machine.
At least that is my theory, I don’t really have any way to test it and I’m sure it will vary depending on what you applications does, how many applications run on the same machine etc etc.