Instruction Per Second (IPS) is the measure of how many times the callback can be called in one second per benchmarking scaffold.
Parallel means that multiple benchmarking scaffolds can be run simultaneously, each are still distinct.
If you want to test parallelism then you set Benchee to no parallelism (1) and you create the parallelism inside your test structure. This is pretty universal among all benchmarkers in all languages.
Setup the connection in setup, benchmark parallel connections within the benchmark (an input
is great for this so you can test different concurrency levels). Do not have Benchee run in parallel. In general never every run Benchee in parallel except for memory testing.
Precisely, because Benchee’s Parallelism is for running tests faster but more inaccurately, it is not to ‘test’ parallelism.
I prefer to setup load in the setup call, much more control. ^.^
Read above in my post here.
Both correct.
You don’t, just absolutely don’t run Benchee in parallel mode for accurate timing measurements, it’s for other purposes.
Not misleading, it is reporting precisely what it should be reporting. If it reported anything else, like 10
, it would be absolutely outright wrong.
This is the absolute right answer.
Given that Ecto is doing type conversions, pool handling, etc… This seems quite accurate.