You can use GitHub - michalmuskala/decompile · GitHub to take a look how things will look at lower levels.

I see, thanks for this. What I found out: It does not compile to the same byte code.

The difference between these two approaches is the difference between a big case statement and a Enum.member? call. It would be cool to see a test with benchee | Hex

My gut feeling is the generated function is faster.

in/2 doesn’t compile to an Enum.member?/2 call when used in a guard:

Nice, should have looked at the documentation

As mentioned above, it would be easy enough to set up a quick benchmark! I’d be interested in the result.

My hypothesis is that performance will be, for all intents and purposes, equivalent. I don’t believe multiple function clause pattern matching does anything fancy that allows it to skip a significant number of clauses here, so both cases involve a linear search.

The first version, using guards, is certainly more idiomatic.

Will do.

So I wrote the following simple Elixir script that uses benchee. I did not give it much thought to be honest. Let me know what you think.

TLDR; Performance for both approaches indeed seem to be equivalent.

Script

defmodule Test do
  @numbers Range.to_list(1..1000)

  def work(input, fun) do
    Enum.map(input, fun)
  end

  for number <- @numbers do
    def do_work_generated(unquote(number)), do: unquote(number)
  end

  def do_work_guards(number) when number in @numbers, do: number
end

Mix.install([:benchee, :benchee_markdown])
list_of_random_numbers = fn -> Enum.map(1..500_000, fn _ -> :rand.uniform(1000) end) end

inputs =
  list_of_random_numbers
  |> Stream.repeatedly()
  |> Stream.take(5)
  |> Stream.with_index()
  |> Stream.map(fn {numbers, idx} -> {"Batch #{idx + 1}", numbers} end)

Benchee.run(
  %{
    "Generated Functions" => fn input -> Test.work(input, &Test.do_work_generated/1) end,
    "Guards" => fn input -> Test.work(input, &Test.do_work_guards/1) end
  },
  memory_time: 60,
  inputs: inputs,
  formatters: [
    {Benchee.Formatters.Markdown, file: "BENCHMARK.md"},
    Benchee.Formatters.Console
  ]
)

Output

Configuration

Benchmark suite executing with the following configuration:

Statistics

Input: Batch 1

Run Time

Run Time Comparison

Memory Usage

Input: Batch 2

Run Time

Run Time Comparison

Memory Usage

Input: Batch 3

Run Time

Run Time Comparison

Memory Usage

Input: Batch 4

Run Time

Run Time Comparison

Memory Usage

Input: Batch 5

Run Time

Run Time Comparison

Memory Usage

Looks like the performance, for both options, is pretty much the same.

I extended this into a couple more cases that got unexpected results:

defmodule Test do
  @numbers Enum.to_list(1..1000)

  @numbers_as_map Enum.to_list(1..1000) |> Map.new(fn el -> {el, true} end)

  def work(input, fun) do
    Enum.map(input, fun)
  end

  for number <- @numbers do
    def do_work_generated(unquote(number)), do: unquote(number)
  end

  def do_work_guards(number) when number in @numbers, do: number

  def do_work_map(number) do
    if Map.has_key?(@numbers_as_map, number) do
      number
    else
      raise "oh no"
    end
  end

  def do_work_in(number) do
    if number in @numbers do
      number
    else
      raise "oh no"
    end
  end

  def do_work_nothing(number) do
    number
  end
end

Mix.install([:benchee, :benchee_markdown])
list_of_random_numbers = fn -> Enum.map(1..500_000, fn _ -> :rand.uniform(1000) end) end

inputs =
  list_of_random_numbers
  |> Stream.repeatedly()
  |> Stream.take(5)
  |> Stream.with_index()
  |> Stream.map(fn {numbers, idx} -> {"Batch #{idx + 1}", numbers} end)

Benchee.run(
  %{
    "Generated Functions" => fn input -> Test.work(input, &Test.do_work_generated/1) end,
    "Guards" => fn input -> Test.work(input, &Test.do_work_guards/1) end,
    "Map" => fn input -> Test.work(input, &Test.do_work_map/1) end,
    "In" => fn input -> Test.work(input, &Test.do_work_in/1) end,
    "Nothing" => fn input -> Test.work(input, &Test.do_work_nothing/1) end,
  },
  memory_time: 60,
  inputs: inputs,
  formatters: [
    {Benchee.Formatters.Markdown, file: "BENCHMARK.md"},
    Benchee.Formatters.Console
  ]
)

The important bits:

• do_work_map uses Map.has_key? to check for membership
• do_work_in uses in in an if (instead of a guard) to check for membership
• do work_nothing doesn’t bother checking at all

My absolute numbers aren’t relevant (my machine is pretty slow, and I’m on 1.13/OTP24), but the differences are peculiar:

• do_work_map is roughly 2x slower than either the guard or the generated cases. That’s surprising, given that map lookup should take far fewer comparisons than simple linear search.
• even weirder: do_work_nothing doesn’t always WIN somehow. It doesn’t ever lose by much, but surely whatever logic is dispatching the function heads or guards should take some time
• do_work_in is MASSIVELY slower, by a factor of 30x.

I’ve tried a few obvious things that seemed like they could be causing slowdown - for instance, not having a raise in the body of do_work_map to see if the optimizer is giving up because an exception might be raised - but none of them have yielded a change.