Surprising beahavior with streams and exceptions

Questions / Help
,
1

Hello everyone, I hope you are all well!

I recently had a WTF moment when working with the Stream module. I’d like to find out if you are equally surprised or if you think that this should be expected behavior. Here’s the code:

Stream.resource(
  fn -> 0 end,
  fn
    5 ->
      IO.puts("source is about to raise")
      raise "resource raised"

    counter ->
      {[counter], counter + 1}
  end,
  fn _ -> IO.puts("source is finishing") end
)
|> Stream.transform(
  fn -> nil end,
  fn n, _ ->
    IO.puts("received #{n}")
    {[n], nil}
  end,
  fn _ -> IO.puts("transform is finishing") end
)
|> Stream.run()

When I run this code, I would expect to see this output:

received 0
received 1
received 2
received 3
received 4
source is about to raise
** (RuntimeError) resource raised
    stream.exs:6: anonymous fn/1 in :elixir_compiler_0.__FILE__/1
    (elixir 1.12.1) lib/stream.ex:1531: Stream.do_resource/5
    (elixir 1.12.1) lib/stream.ex:880: Stream.do_transform/5
    (elixir 1.12.1) lib/stream.ex:649: Stream.run/1

Why? Because when the source raises before emitting element n = 5, IMO that should be the end of the stream, and no downstream consumers should be executed anymore.

Instead, what I see is the following:

received 0
received 1
received 2
received 3
received 4
source is about to raise
source is finishing
transform is finishing
** (RuntimeError) resource raised
    stream.exs:6: anonymous fn/1 in :elixir_compiler_0.__FILE__/1
    (elixir 1.12.1) lib/stream.ex:1531: Stream.do_resource/5
    (elixir 1.12.1) lib/stream.ex:880: Stream.do_transform/5
    (elixir 1.12.1) lib/stream.ex:649: Stream.run/1

As you can see, before the exception is raised, the finishing functions of both the source and the transform step are called. This implies that the exception has been caught somewhere and reraised after these finishing functions are executed.

This behavior has puzzled me when I first encountered it. Do you share my feeling or do you think that this should be the normal behavior, and if so, why?

Thanks!
Max

2

I can’t comment on whether or not this behaviour is desirable, but it is expected according to the docs: Stream — Elixir v1.16.0. It says

Similar to transform/3 but the initial accumulated value is computed lazily via start_fun and executes an after_fun at the end of enumeration (both in cases of success and failure).

(Emphasis mine).

2 Likes
3

Thanks @John-Goff . I had missed that part of the docs. I still find this behavior counterintuitive - I would like to know why the decision was made to implement it this way - but since it’s documented, it should at least not come as a surprise.