User initiated cancellation of a chunked download.

    Stream.iterate(initial_page(url, auth, pit_id), &next_page(url, auth, &1))
    |> Stream.take_while(&(&1 != :eof))   
    |> CSV.dump_to_stream()
    |> Stream.each(&chunk(conn, &1))

Hi, I’m building a csv by using Stream.iterate() to paginate over elastic search responses with their “search_after” functionality and chunking each page as csv rows. All of this works really well, but I can’t figure out how to handle a user initiated cancellation. I’ve tried using reduce_while as shown in the chunk/2 documentation but the error tuple never evaluates. I assume there’s some process that’s just ending but where would it’s shutdown message go?

The reason this is so important is elasticsearch highly recommends killing a PIT (point in time) when you’re finished with it as they are extremely expensive, I’d like to just run a callback whenever a user cancels the csv download.

Which networking library are you using to fetch data from Elastic?

I’m using Req, and passing the result of!(body: Jason.encode!(query)) to Stream.iterate