Hi,

I am trying to stream a file from AWS.
It is a very large file, so I thought it would be better to use Stream and , once that works, to chunk the data.
I am using NimbleCSV - the parse_stream function.
I am getting back an empty list after running Enum.to_list

This is my code so far:

alias NimbleCSV.RFC4180, as: CSV

ExAws.S3.download_file("testfiles", "file_upload.csv", :memory) 
|> ExAws.stream!() 
|> CSV.parse_stream 
|> Enum.to_list

I would like to do something like:

alias NimbleCSV.RFC4180, as: CSV

ExAws.S3.download_file("testfiles", "file_upload.csv", :memory) 
|> ExAws.stream!() 
|> CSV.parse_stream 
|> Stream.chunk_every(10_000)
|> Stream.map(& process_stream(&1)
|> Enum.to_list

Thanks!