zwippie

zwippie

Nimble_csv dump with each column quoted

I want to dump some data to a CSV file where each column value is separated with a semicolon and wrapped inside double quotes. The semicolon is no problem, but the double quotes are.

This does not work, no double quotes:

NimbleCSV.define(MyParser, separator: ";", escape: "\"")
data = [["firstname", "lastname"], ["Foo", "Bar"]]
IO.iodata_to_binary MyParser.dump_to_iodata(data)

That will return:

"firstname;lastname\nFoo;Bar\n"

But i want this:

"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""

How to wrap each column with double quotes?

Marked As Solved

peerreynders

peerreynders

iex(1)> data = [["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
[["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)
"\"\"\"firstname\"\"\";\"\"\"lastname\"\"\"\n\"\"\"Foo\"\"\";\"\"\"Bar\"\"\"\n"
iex(3)> 

… so not exactly.

rfc4180

  1. Each field may or may not be enclosed in double quotes (however
    some programs, such as Microsoft Excel, do not use double quotes
    at all). If fields are not enclosed with double quotes, then
    double quotes may not appear inside the fields.

NimbleCSV has no problem consuming

"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""

it just doesn’t seem interested in producing it (likely to avoid unnecessarily bloating the output).

iex(1)> MyParser.parse_string("\"firstname\";\"lastname\"\n\"Foo\";\"Bar\"",[headers: false])
[["firstname", "lastname"], ["Foo", "Bar"]]
iex(2)> 
  1. Fields containing line breaks (CRLF), double quotes, and commas
    should be enclosed in double-quotes.

NimbleCSV will only use the double quotes when absolutely necessary, e.g.:

iex(1)> data = [["firstname", "lastname"], ["Fo\no", "Bar"]]
[["firstname", "lastname"], ["Fo\no", "Bar"]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)   
"firstname;lastname\n\"Fo\no\";Bar\n"
iex(3)> data = [["firstname", "lastname"], ["Fo\"o", "Bar"]]
[["firstname", "lastname"], ["Fo\"o", "Bar"]]
iex(4)> IO.iodata_to_binary MyParser.dump_to_iodata(data)   
"firstname;lastname\n\"Fo\"\"o\";Bar\n"
iex(5)> 

that last example also demonstrating

  1. If double-quotes are used to enclose fields, then a double-quote
    appearing inside a field must be escaped by preceding it with
    another double quote.

Also Liked

zwippie

zwippie

Thank you for your great explanation! Considering rfc4180, it all makes sense of course. I would welcome a force_escape option for NimbleCSV though.

Now I’m going to find out how many clients rely on our current (quoted) CSV output format.

nallwhy

nallwhy

Simple implementation

Where Next?

Popular in Questions Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

We're in Beta

About us Mission Statement