For Comprehension: there's no assignment, only filters?

Chat / Discussions
1

Hi all, recently i have been trying out for comprehension after reading Introducing `for let` and `for reduce` and realizing i am also have severly underutilized for. Just now i have spend quite sometime debugging code written with for only to realize there is the following: there’s no “only” binding operation, it always also get used with filters.

For example, consider following code:

data = [%{a: 1, b: 2}, %{a: 2}]

for datum <- data,
  b = datum[:b],
  b == nil do
    datum
end

what would you expect the result be? [%{a: 2}] like i was? the actual result (on elixir 1.10.3) was []. b = datum[:b] is actually treated as filter, and elixir recognize nil and false as falsey value.

I think this is pretty strange behaviour, though maybe this is the implication on A comprehension is made of three parts: generators, filters, and collectables. from https://elixir-lang.org/getting-started/comprehensions.html ?

For context, even at the web tutorial we find following code, in which path = Path.join(dir, file) is treated as variable assignment.

dirs = ['/home/mikey', '/home/james']

for dir <- dirs,
    file <- File.ls!(dir),
    path = Path.join(dir, file),
    File.regular?(path) do
  File.stat!(path).size
end

and at Haskell, they have separation between declaration and filter, like in following

image
image1627×236 17.7 KB

1 Like
2

That‘s how it‘s supposed to work. Assignments evaluate to the value of the left hand side (left == (a = left)). If that value is falsy the current generated value is filtered.

That doesn‘t mean you cannot use assignments, but you need to be careful with ones, which can return falsy values.

3

Given the original comment and @LostKobrakai 's response, I’m thinking something like this would achieve what you want:

(Modifying the original code snippet.)

data = [%{a: 1, b: 2}, %{a: 2}]

for datum <- data,
  {:ok, b} = {:ok, datum[:b]},
  b == nil do
    datum
end
4

I understand that b = datum[:b] is an expression and it would return the value of datum[:b], what i want to highlight is that assignment is common, and having it treated always as filter result in really suprising gotcha, and you would need to be really careful everywhere as faulty value like nil and false is commonly used (ex: Ecto get_entity function has possibility return nil, Map.get default to nil, access pattern [:b] default to nil).

That’s true, wrapping it inside {:ok, } tuple does solve the issue, but in spirit of idiomatic code, it looks like more noisy compared to

for datum <- data,
  b = datum[:b],
  b == nil do
    datum
end

in spirit of idiomatic code, wouldn’t it better to make for treat b = datum[:b] as purely assignment operator? though i understand it probably will create breaking change. alternatively, maybe making a macro let which would transform let(b = datum[:b]) to {:ok, b} = {:ok, datum[:b]} would be more feasible?

example if there’s let

for datum <- data,
  let(b = datum[:b]),
  b == nil do
    datum
end
5

Just realize it, but probably only {} would work too, still thinking having let would be better though:

data = [%{a: 1, b: 2}, %{a: 2}]

for datum <- data,
  {b} = {datum[:b]},
  b == nil do
    datum
end

or

data = [%{a: 1, b: 2}, %{a: 2}]

for datum <- data,
  {b = datum[:b]},
  b == nil do
    datum
end
6

In your case you could also do:

for datum when not is_map_key(datum, :b) <- data do
  datum
end
1 Like
7

thanks for pointing that out!

what do you think about having an assignment syntax in for which are not used as filter though? should wrapping assignment in {} be default practice when using for comprehension?

8

In such cases I also tend to do this:

for x <- list,
    y <- [may_be_nil(x)],
    do: y

So that is likely the approach I would go with if it can’t be done with a guard?

2 Likes
9

That’s interesting, thinking assignment as a special case of generator. Should i update elixir tutorial at Comprehensions - The Elixir programming language (and maybe other related docs) to this syntax (and maybe add warning about using = in for?)

dirs = ['/home/mikey', '/home/james']

for dir <- dirs,
    file <- File.ls!(dir),
    path <- [Path.join(dir, file)],
    File.regular?(path) do
  File.stat!(path).size
end
10

I don’t think it is worth doing this change in general, because it can be nil in many cases and sometimes you want to discard nil values, but we probably need to add a note to the API documentation just in case.

3 Likes