Ideal way to use stream

One of the advantages of using stream is do that it does not create intermediate list for each function
For example, this will create 2 intermediate lists

list
|> Enum.map(&foo/1) # intermediate list
|> Enum.map(&bar/1) # final list

By do so, we avoid having intermediate list and have an improvement in performance

list
|> Stream.map(&foo/1) 
|> Stream.map(&bar/1) 
|> Enum.to_list() # final list

Q: This is shorter code in comparison but by doing so are we still creating intermediate lists?

list
|> Stream.map(&foo/1)  # <--- ???
|> Enum.map(&bar/1) # final list

No, in your final example there is no intermediate list created.

However, you should test both, because there are overheads involved when using Stream so while you get lazy evaluation, it might not actually be faster depending on the situation.

For example, in this case, the Enum version is faster on my machine:

defmodule Example do
  def enum do
    1..10_000
    |> Enum.map(fn x -> x ** 2 end)
    |> Enum.map(fn x -> x / 2 end)
  end

  def stream do
    1..10_000
    |> Stream.map(fn x -> x ** 2 end)
    |> Enum.map(fn x -> x / 2 end)
  end

  def benchmark do
    Benchee.run(%{
      "enum" => fn -> enum() end,
      "stream" => fn -> stream() end
    })
  end
end
Comparison:
enum          1.45 K
stream        0.88 K - 1.65x slower +0.45 ms

Operating System: macOS
CPU Information: Apple M1 Pro
Number of Available Cores: 10
Available memory: 16 GB
Elixir 1.13.4
Erlang 24.3.4

4 Likes

If you want performance that should be faster though:

Enum.map(list, & &1 |> foo() |> bar())

Edit: as an equivalent of using Stream. Sometimes you want to build an intermiediary list after each step, notably when there are side effects, in order to be sure all items pass the first step before going on to the next.