aura999

aura999

Issue with streaming into a csv

I am trying to stream into a csv file using nimble_csv(1.2.0) but running into issues.

This is the code I am using to create a csv file from the stream.

defmodule Utils
  alias NimbleCSV.RFC4180, as: CSV

  def write_to_file(content_stream) do
    path = Path.join(System.tmp_dir(), "#{UUID.uuid4(:hex)}.csv")

    content_stream
    |> CSV.to_line_stream()
    |> CSV.parse_stream([skip_headers: false])
    |> Stream.into(File.stream!(path, [:write, :utf8]))
    |> Stream.run()
  end
end

This is my test data

contents = [
    "First Name,Last Name,Email\n",
    "David,Byrne,david@test.com"
  ]
  |> Stream.map(&String.trim_leading/1)
   
Utils.write_to_file(contents)

The file is created however the contents in the file are missing the comma separator
and the new line characters

First NameLast NameEmailDavidByrnedavid@test.com

If however I update Utils.write_to_file/1 to by removing parse_stream/2

def write_to_file(content_stream) do
    path = Path.join(System.tmp_dir(), "#{UUID.uuid4(:hex)}.csv")

    content_stream
    |> CSV.to_line_stream()
    |> Stream.into(File.stream!(path, [:write, :utf8]))
    |> Stream.run()
end

I get the csv contents in the correct format.

First Name,Last Name,Email
David,Byrne,david@test.com

I am not sure if I am missing any options that I need to pass to parse_stream/2 that would cause the contents to be malformed.

Any help would be appreciated

Marked As Solved

LostKobrakai

LostKobrakai

Streams are lazy enumerables. Being a stream doesn’t tell you anything about what data the streams deals with. parse_stream maps from a stream of csv formatted binaries to a stream of rows being lists of cell contents. It’s the lazy version to parse_enumerable, which immediately returns [[binary()]].

In your case I’m really confused though because if you already have a csv file and you want to write that csv file, then there’s no need for NimbleCSV in the first place.

Enum.into(content_stream, file_collectable) would be all you need.

What makes streams even more confusing here is that a File.Stream struct implements both Enumerable.t (a - in this case lazy – collection to enumerate) as well as Collectable.t (a collection, which items can be pushed into), which work independently. You’re using the latter functionality here.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement