Improving single item filters in `for`

Chat / Discussions
1

I’m finding myself often using the following pattern with for:

for item <- listing,
    result = validate_something_about_item(item),
    match?({:ok, _}, result),
    {:ok, data} = result do
  # do something with data
end

This works, but at the same time is a super verbose way of handling the task.

It would be great to have e.g. <~ for the iterating part of for, <- to work similar to with in that it tries a pattern match an value directly (no iteration) and just goes to the next iteration if not matching:

for item <~ listing,
    {:ok, data} <- validate_something_about_item(item) do
  # do something with data
end

I’m aware this is not going to happen like that because it’s a breaking change. Also I wouldn’t want to propose the actually backwards compatible alternative of switching <~/<- semantic, as I feel that would further create confusion. Beginners seem to already be confused why <- does sometimes act on lists and sometimes on single items.

So I’m wondering if instead there could be something like match?(), which does however actually create bindings:

for item <- listing,
    skip?({:ok, data}, validate_something_about_item(item)) do
  # do something with data
end

I’d also be open to other suggestions of handling that I might not have though of.

3 Likes
2

This seems to do what you’re looking for:

# NOTE: the only part of this that's important is returning {:ok, _} vs {:error, _}
defmodule Validator do                                  
  def validate(<<x::binary-size(3)>>), do: {:ok, String.upcase(x)}  
  def validate(x), do: {:error, x}
end

a = ["foo", "bar", "mumble"]

for x <- a, {:ok, arg} <- [Validator.validate(x)], do: arg        
# returns ["FOO", "BAR"]
3

While this does work, wrapping things in a list just to pull it directly out again kind feels wrong.

4

From the documentation of for:

Generators can also be used to filter as it removes any value that doesn’t
match the pattern on the left side of <-:

So what about:

for {:ok, data} <- Enum.map(listing, &validate_something/1) do
  # ...
end
1 Like
5

Another possible workaround, but it will result in one more iteration over the list.

for i <- 1..10, rem(i, 2) == 1, do: i

This will iterate only once filtering out the odd i's. E.g. Enum.filter_map has been deprecated with the hint of using for instead.

6

Very interesting!
I did a couple of tests by writing the same function multiple times in different styles (Enum, for and :lists; I did not check Erlang’s list comprehensions) and see what kind of bytecode they would compile down to.

The conclusion is that for seems oddly enough to be slightly more optimized in that the body of the for-loop is inlined.

It also means that currently the BEAM does not perform any kind of list fusion. This is an optimization that might be added to the compiler in the future for sure, as it is relatively straightforward.

Something to think about right now is if you should care for most application code about traversing a list twice. If the list is short the difference is negligible. If the list is long, you probably are better off using Stream instead anyway. I’d suggest to opt for a pipeline of Enum-functions until profiling/benchmarking shows that that particular piece of code is too slow for what it is intending to do, at which time it could be rewritten with direct calls to the functions in the:lists module or potentially manual recursion.

I also would like to point out that the current warning that is shown when you are using Enum.filter_map is

Enum.filter_map/3 is deprecated. Use Enum.filter/2 + Enum.map/2 or for comprehensions instead

, hinting at no particular preference of either for or Enum.