Error handling with streams?

Chat / Discussions
1

is there a proper way to handle errors with streams?
If I use Stream.resource/3 to create a stream that consumes external resource subject
to say network errors is there a good way to handle that situation?

1 Like
2

anyone :slight_smile: ?

3

Depends on the kind of network error.

Many network errors just need to be retried. ExAws for example will retry a configurable number of times with various backoff amounts for network errors.

If retries fail or the error is unrcoverable the convention is to just crash. You can wrap the stream in a try construct if you want to take particular action.

2 Likes
4

Cool thank you Ben that what I was doing (retry n times with increasing backoff time) then crash, but was not sure if there is some other more useful convention :).

5

By crashing you mean you raise/1 in next_fun?:

fn params ->
  case run_some_search_query(params) do
    {:error, reason} = error ->
      raise(error)

    [] ->
      {:halt, "No more results, done."}

    result_list ->
      (continue streaming)
  end
end,