If I run the stream as Stream.run will it run all the chunks until the stream is all streamed?
All chunks. But try it. 
When will the temp streamed uploaded file get deleted?
Uploaded files are saved to a temporary directory and deleted afterwards, yes.
Thanks, all chunks got streamed now, but I could not parse them, here is what I got as a sample chunk from |> Stream.chunk(10)
1AntonioMartinezfemeise@bi.gov3484293762RobertButleramfob@vaim.io7388480513GordonSimonnak@jodcon.org8829569684TylerFlowersid@do.net5789928675LandonRossriojipof@fuju.gov7791899366EthanChavezozeketja@vucocabo.gov7585680127AmyHendersoneruraep@emu.io9750233448PaulWilliamsabmahu@bi.edu7778078969TomHerreraosaik@alaigocuc.com45881961610HelenCurrymuw@pi.io598349524
While NimbleCSV will generate an array for each line in the CSV file, |> Stream.chunk(10) has concatenated the arrays as above, making it impossible to get the data using pattern matching.
So, what’s the advice here?
That’s what Stream.chunk(10) does, it builds chunks up to 10 elements. Therefore inside Task.async_stream, you will have list of 10 elements, then you need to use Enum.each or Enum.map and friends to traverse them, process them, and insert them into the database in chunks.
Thanks, I’ve finally implemented it!, but I have to admit, that I had to change my mindset to do it
Here is a sample chunk:
[%{first: “Antonio”, seq: “1”}, %{first: “Robert”, seq: “2”},
%{first: “Gordon”, seq: “3”}, %{first: “Tyler”, seq: “4”},
%{first: “Landon”, seq: “5”}, %{first: “Ethan”, seq: “6”},
%{first: “Amy”, seq: “7”}, %{first: “Paul”, seq: “8”},
%{first: “Tom”, seq: “9”}, %{first: “Helen”, seq: “10”}]
As I have used Stream.map in the pipeline:
File.stream!(path)
|> MyParser.parse_stream
|> Stream.map(fn n ->
case n do
[seq,first,last,email,digit] -> %{seq: seq, first: first}
_ -> ""
end
end)
|> Stream.chunk(10)
|> Task.async_stream(&MyApp.CsvsController.chunk_handler_fn/1)
|> Stream.run
Thanks all for support 
