zwippie
Nimble_csv dump with each column quoted
I want to dump some data to a CSV file where each column value is separated with a semicolon and wrapped inside double quotes. The semicolon is no problem, but the double quotes are.
This does not work, no double quotes:
NimbleCSV.define(MyParser, separator: ";", escape: "\"")
data = [["firstname", "lastname"], ["Foo", "Bar"]]
IO.iodata_to_binary MyParser.dump_to_iodata(data)
That will return:
"firstname;lastname\nFoo;Bar\n"
But i want this:
"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""
How to wrap each column with double quotes?
Marked As Solved
peerreynders
iex(1)> data = [["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
[["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)
"\"\"\"firstname\"\"\";\"\"\"lastname\"\"\"\n\"\"\"Foo\"\"\";\"\"\"Bar\"\"\"\n"
iex(3)>
… so not exactly.
- Each field may or may not be enclosed in double quotes (however
some programs, such as Microsoft Excel, do not use double quotes
at all). If fields are not enclosed with double quotes, then
double quotes may not appear inside the fields.
NimbleCSV has no problem consuming
"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""
it just doesn’t seem interested in producing it (likely to avoid unnecessarily bloating the output).
iex(1)> MyParser.parse_string("\"firstname\";\"lastname\"\n\"Foo\";\"Bar\"",[headers: false])
[["firstname", "lastname"], ["Foo", "Bar"]]
iex(2)>
- Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes.
NimbleCSV will only use the double quotes when absolutely necessary, e.g.:
iex(1)> data = [["firstname", "lastname"], ["Fo\no", "Bar"]]
[["firstname", "lastname"], ["Fo\no", "Bar"]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)
"firstname;lastname\n\"Fo\no\";Bar\n"
iex(3)> data = [["firstname", "lastname"], ["Fo\"o", "Bar"]]
[["firstname", "lastname"], ["Fo\"o", "Bar"]]
iex(4)> IO.iodata_to_binary MyParser.dump_to_iodata(data)
"firstname;lastname\n\"Fo\"\"o\";Bar\n"
iex(5)>
that last example also demonstrating
- If double-quotes are used to enclose fields, then a double-quote
appearing inside a field must be escaped by preceding it with
another double quote.
Also Liked
zwippie
Thank you for your great explanation! Considering rfc4180, it all makes sense of course. I would welcome a force_escape option for NimbleCSV though.
Now I’m going to find out how many clients rely on our current (quoted) CSV output format.
nallwhy
Simple implementation
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








