`Stream.flat_map` influences the behaviour of preceeding `Stream` operations in a pipe

Questions / Help
1

This is a repost from the mailing list, as i realized only after sending it there that the mailing list is depreciated.

Today i banged my head against this one :

iex(13)> [[true,false],[false],[true]]|>Stream.filter(fn([x|tail])-> IO.inspect([filter: x]); x end) 
  |>Stream.take(1) 
  |> Stream.flat_map(fn(x)-> IO.inspect([flat_map: x]); x end)  
  |> Enum.to_list()
[filter: true]
[flat_map: [true, false]]
[filter: false]
[filter: true]
[true, false]

Why is Stream.filter’s argument executed after Stream.flat_map’s argument has been called?

taking away the culprit Stream.flat_map prevents the execution of Stream.filter’s function after the first element:

[[true,false],[false],[true]]|>Stream.filter(fn([x|tail])-> IO.inspect([filter: x]); x end) 
  |>Stream.take(1) 
  |> Enum.to_list()
[filter: true]
[[true, false]]

It seems to me that this is a bug, but what i find more puzzling is that i do not know how Stream.flat_map has even any influence on the Stream.filter function.

Did i miss here how Stream.flat_map should work or shall I file ~a bug request~ [EDIT: an issue :)]?

[Edit 2:] Actually the “culprit” is Stream.take/2. Citing @josevalim from the mailing list:

You got the bug backwards. The issue is that flat_map continues even after take(1). Streams are by design meant to halt when items will no longer be consumed.

2 Likes
2

Here is a more minimal example that demonstrates the behaviour:

 0..10
|> Stream.chunk(2)
|> Stream.map(&IO.inspect/1)
|> Stream.take(2)
|> Stream.flat_map(&(&1))
|> Enum.to_list

[0, 1]
[2, 3]
[4, 5]
#=> [0, 1, 2, 3]
1 Like
3

Yes [more minimal], but from your example it is not obvious that [4,5] is actually mapped after Stream.flat_map's argument has been evaluated.

1 Like
4

This has been fixed in Elixir master.

5 Likes
5

Thank you! For anybody reading, this is the gh commit solving this: 039619

1 Like