Benchmark with benchee having separate inputs for same functions

Questions / Help
, , ,
1

Hi, i am trying to benchmark a function with different input but the results are getting shown in different HTML files.

my code:

map_fun = fn i -> [i, i * i] end

Benchee.run(
  %{
    "10 values" => fn input -> Enum.flat_map(input, map_fun) end,
    "20 values" => fn input -> Enum.flat_map(input, map_fun) end,
    "30 values" => fn input -> Enum.flat_map(input, map_fun) end
  },
  inputs: %{
    :ten => 1..10,
    :twenty => 1..20,
    :thirty => 1..30
  },
  time: 10,
  memory_time: 30,
  formatters: [
    {Benchee.Formatters.HTML, file: "samples_output/ex/ex.html"},
    {Benchee.Formatters.Console, extended_statistics: true}
  ],
  warmup: 30
)

I am getting results but the input :ten gets applied to all three functions in Benchee.run. All I want is to specify which input is for which run. Can I somehow do that with pattern matching?

Also, I don’t want to declare the list above Benchee.run, how can I do it with the help of inputs.

Thanks! :smiley:

2

Not an expert on Benchee, but if you only want to benchmark one function, I think it should only be once in Benchee.Run, and it will run on the different inputs, like so:

3

but this don’t compare the results. It just display them separately.

4

In this case, you want to just put the inputs in the function and not use the inputs feature. That’s designed for testing the same inputs against different functions, not for testing the same function against different inputs.

map_fun = fn i -> [i, i * i] end

Benchee.run(
  %{
    "10 values" => fn input -> Enum.flat_map(0..10, map_fun) end,
    "20 values" => fn input -> Enum.flat_map(0..20, map_fun) end,
    "30 values" => fn input -> Enum.flat_map(0..30, map_fun) end
  },
  time: 10,
  memory_time: 30,
  formatters: [
    {Benchee.Formatters.HTML, file: "samples_output/ex/ex.html"},
    {Benchee.Formatters.Console, extended_statistics: true}
  ],
  warmup: 30
)
3 Likes