Yeah it looks like the compiler is able to optimize better if facts are known at compile time.
But here’s something interesting: I have adapted the code a bit because eventually, I’m gonna be working with strings. And if we add binary pattern matching to the mix, performance seems to be different.
Side note: Reading the documentation of Kernel.in/2 I read the following:
However, this construct will be inefficient for large lists. In such cases, it is best to stop using guards and use a more appropriate data structure, such as
MapSet.
I therefore added a test using MapSets (which, afaict, is equivalent to @al2o3cr’s test using is_map_key).
TLDR;
In this case, guards seem to make the race, being twice as fast as generated functions and 2.5x faster than MapSets
Test
defmodule Test do
@chars Range.to_list(1000..1)
@chars_map_set MapSet.new(1000..1)
def work(input, fun) do
Enum.map(input, fun)
end
for char <- @chars do
def do_work_generated(<<unquote(char)::utf8>>), do: unquote(char)
def do_work_generated(<<unquote(char)::utf8, rest::binary>>) do
do_work_generated(rest)
end
end
def do_work_guards(<<char::utf8>>) when char in @chars, do: char
def do_work_guards(<<char::utf8, rest::binary>>) when char in @chars do
do_work_guards(rest)
end
def do_work_mapset(<<char::utf8>>) do
if MapSet.member?(@chars_map_set, char) do
char
else
:error
end
end
def do_work_mapset(<<char::utf8, rest::binary>>) do
if MapSet.member?(@chars_map_set, char) do
do_work_mapset(rest)
else
:error
end
end
end
Mix.install([:benchee, :benchee_markdown])
list_of_random_strings = fn ->
Enum.map(1..500_000, fn _ ->
for _ <- 1..10,
into: "",
do: <<Enum.random(Range.to_list(?0..?9) ++ Range.to_list(?A..?z))>>
end)
end
inputs =
list_of_random_strings
|> Stream.repeatedly()
|> Stream.take(5)
|> Stream.with_index()
|> Stream.map(fn {strings, idx} -> {"Batch #{idx + 1}", strings} end)
Benchee.run(
%{
"Generated Functions" => fn input -> Test.work(input, &Test.do_work_generated/1) end,
"Guards" => fn input -> Test.work(input, &Test.do_work_guards/1) end,
"MapSet" => fn input -> Test.work(input, &Test.do_work_mapset/1) end
},
memory_time: 60,
inputs: inputs,
parallel: 10,
formatters: [
{Benchee.Formatters.Markdown, file: "BENCHMARK_STRINGS.md"},
Benchee.Formatters.Console
]
)
Output
Statistics
Statistics
Input: Batch 1
Run Time
| Name | IPS | Average | Devitation | Median | 99th % |
|---|---|---|---|---|---|
| Guards | 31.07 | 32.19 ms | ±8.67% | 32.09 ms | 46.84 ms |
| Generated Functions | 15.54 | 64.36 ms | ±5.44% | 63.96 ms | 90.67 ms |
| MapSet | 10.99 | 91.02 ms | ±5.33% | 90.73 ms | 123.52 ms |
Run Time Comparison
| Name | IPS | Slower |
|---|---|---|
| Guards | 31.07 | |
| Generated Functions | 15.54 | 2.0x |
| MapSet | 10.99 | 2.83x |
Memory Usage
| Name | Average | Factor |
|---|---|---|
| Guards | 26.70 MB | |
| Generated Functions | 26.70 MB | 1.0x |
| MapSet | 26.70 MB | 1.0x |
Input: Batch 2
Run Time
| Name | IPS | Average | Devitation | Median | 99th % |
|---|---|---|---|---|---|
| Guards | 31.57 | 31.68 ms | ±8.91% | 31.97 ms | 45.93 ms |
| Generated Functions | 15.64 | 63.94 ms | ±5.55% | 64.00 ms | 91.14 ms |
| MapSet | 11.10 | 90.06 ms | ±4.90% | 90.00 ms | 119.82 ms |
Run Time Comparison
| Name | IPS | Slower |
|---|---|---|
| Guards | 31.57 | |
| Generated Functions | 15.64 | 2.02x |
| MapSet | 11.10 | 2.84x |
Memory Usage
| Name | Average | Factor |
|---|---|---|
| Guards | 26.70 MB | |
| Generated Functions | 26.70 MB | 1.0x |
| MapSet | 26.70 MB | 1.0x |
Input: Batch 3
Run Time
| Name | IPS | Average | Devitation | Median | 99th % |
|---|---|---|---|---|---|
| Guards | 31.43 | 31.81 ms | ±8.78% | 31.89 ms | 46.68 ms |
| Generated Functions | 15.40 | 64.95 ms | ±5.93% | 64.84 ms | 93.51 ms |
| MapSet | 11.04 | 90.54 ms | ±4.83% | 90.49 ms | 118.91 ms |
Run Time Comparison
| Name | IPS | Slower |
|---|---|---|
| Guards | 31.43 | |
| Generated Functions | 15.40 | 2.04x |
| MapSet | 11.04 | 2.85x |
Memory Usage
| Name | Average | Factor |
|---|---|---|
| Guards | 26.70 MB | |
| Generated Functions | 26.70 MB | 1.0x |
| MapSet | 26.70 MB | 1.0x |
Input: Batch 4
Run Time
| Name | IPS | Average | Devitation | Median | 99th % |
|---|---|---|---|---|---|
| Guards | 31.08 | 32.18 ms | ±8.48% | 32.34 ms | 46.62 ms |
| Generated Functions | 15.25 | 65.58 ms | ±6.42% | 65.17 ms | 97.31 ms |
| MapSet | 12.31 | 81.22 ms | ±7.78% | 79.51 ms | 108.75 ms |
Run Time Comparison
| Name | IPS | Slower |
|---|---|---|
| Guards | 31.08 | |
| Generated Functions | 15.25 | 2.04x |
| MapSet | 12.31 | 2.52x |
Memory Usage
| Name | Average | Factor |
|---|---|---|
| Guards | 26.70 MB | |
| Generated Functions | 26.70 MB | 1.0x |
| MapSet | 26.70 MB | 1.0x |
Input: Batch 5
Run Time
| Name | IPS | Average | Devitation | Median | 99th % |
|---|---|---|---|---|---|
| Guards | 29.61 | 33.78 ms | ±8.29% | 33.12 ms | 50.84 ms |
| Generated Functions | 15.13 | 66.08 ms | ±5.99% | 65.88 ms | 97.77 ms |
| MapSet | 11.20 | 89.32 ms | ±53.75% | 82.30 ms | 440.40 ms |
Run Time Comparison
| Name | IPS | Slower |
|---|---|---|
| Guards | 29.61 | |
| Generated Functions | 15.13 | 1.96x |
| MapSet | 11.20 | 2.64x |
Memory Usage
| Name | Average | Factor |
|---|---|---|
| Guards | 26.70 MB | |
| Generated Functions | 26.70 MB | 1.0x |
| MapSet | 26.70 MB | 1.0x |
