Efficient use of `reduce`

Questions / Help
,
1

Imagine I have a simple list

my_list = [1, 2, 3]

and I want to “do something” to each element in a reduce loop, removing elements that error/fail while “doing something”.

It might look like this:

    my_list
    |> Enum.reduce([], fn x, acc ->
      with {:ok, result: result} <- do_something(x) do
        acc ++ [result]
      else
        _ -> acc
      end
    end)

That gets me the result I want.

However, I know that in Elixir it is more efficient to do [result] ++ acc than it is to do acc ++ [result].

So, for large lists, would it be more efficient to use [result] ++ acc in the reduce loop and then in my pipeline use Enum.reverse to put the list back in original order?

2

The most efficient version to do this is probably doing [result|acc] and Enum.reverse after building the list.

4 Likes
3

I suspected that might be the case. I always forget about the [hd | tl] syntax. Thanks!

4

Well, in this case I would use Enum.flat_map/2 instead (which behind the hood will do almost the same thing you are doing there with Enum.reverse/1.

2 Likes
5

Note that the compiler actually replaces [hd] ++ acc with [hd | acc] (see https://erlang.org/doc/efficiency_guide/myths.html#myth--operator--++--is-always-bad) so if you try to profile both versions you might not see any difference.

1 Like
6

This works perfectly, thanks!

7

Alternatively to Enum.flat_map/2 you can use a for-comprehension:

for with 1-element list

for i <- my_list,
    # We have to wrap the `do_something` call in a list to use `<-` for filtering
    {:ok, result: result} <- [do_something(i)] do
  result
end

for with filter clause

for i <- my_list,
    ok_or_error = do_something(i),
    match?({:ok, _}, ok_or_error) do
  {:ok, result: result} = ok_or_error

  result
end
2 Likes
8

This is equally perfect. Too many solutions 0_o