I am using DataMorph to import some csv output into Elixir structs, and I want the output to go to a file I can read into an editor.
This is the example on the page.
File.stream!('./tmp.csv') \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> Enum.to_list
# [
# %OpenRegister.IsoCountry{iso: "nz", name: "New Zealand"},
# %OpenRegister.IsoCountry{iso: "gb", name: "United Kingdom"}
# ]
How do I pipe to a function that will save it to disk?
|> Enum.reduce("", fn(struct, string) -> string <> "iso: #{struct.iso}, name: #{struct.name}\n" end)
will give you
iso: nz, name: New Zealand
iso: gb, name: United Kingdom
I have got that part working. What I need is to save the output to a file. Currently it is just being output in iex.
If you want to stream the whole thing you should be able to do
File.stream!("tmp.csv") \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> Stream.map(fn(struct) -> "iso: #{struct.iso}, name: #{struct.name}\n" end) \
|> Stream.into(File.stream!("output.txt")) \
|> Stream.runWhen I append the &File.write as you suggested this is the error I get
File.stream!('template_params_import.csv') \
|> DataMorph.structs_from_csv("voip-provisioning", :tpl_params) \
|> Enum.to_list \
|> &File.write('template_params_import.key.list', &1)
v
warning: piping into a unary operator is deprecated. You could use e.g. Kernel.+(5) instead of +5
iex:7
** (ArgumentError) cannot pipe Enum.to_list(DataMorph.structs_from_csv(File.stream!('template_params_import.csv'), "voip-provisioning", :tpl_params)) into &(File.write('template_params_import.key.list', &1)), can only pipe into local calls foo(), remote calls Foo.bar() or anonymous functions calls foo.()
(elixir) lib/macro.ex:149: Macro.pipe/3
(stdlib) lists.erl:1263: :lists.foldl/3
(elixir) expanding macro: Kernel.|>/2
iex:7: (file)This is the exact example from the DataMorph page. I need the exact output shown (which is commented) to go into the output file, ie the output printed in iex.
File.stream!('./tmp.csv') \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> Enum.to_list
# [
# %OpenRegister.IsoCountry{iso: "nz", name: "New Zealand"},
# %OpenRegister.IsoCountry{iso: "gb", name: "United Kingdom"}
# ]First reduce to a string as per my first comment, then write to the file. Alternatively, just stream the whole thing.
If you really want the exact output—brackets, percent signs, and all—then you can do
...
|> Enum.to_list \
|> inspect(limit: :infinity) \
|> (&(&1 <> "\n")).() \
|> (&File.write("output.txt", &1)).()Does this code came after the Enum.to_list or after DataMorph.structs_from_csv?
inspect(limit: :infinity) comes after Enum.list. If you don’t need the Elixir syntax to be a part of the output though, I would highly recommend using one of the options.
File.stream!("tmp.csv") \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> Stream.map(fn(struct) -> "iso: #{struct.iso}, name: #{struct.name}\n" end) \
|> Stream.into(File.stream!("output.txt")) \
|> Stream.runThe input is coming from a CSV file so there is no way to tell the exact headers before hand for this line to be generated.
|> Stream.map(fn(struct) -> "iso: #{struct.iso}, name: #{struct.name}\n" end) \
Unless some function is used to read the first line for the headers and generate the equivalent code and structures beforehand.
The option in response no 8 works, but it is all a single line. Will there be some way print each struct on a single line?
Does this mean the code can be applied to any output from the code, ie any thing output by the code to the screen?
Presumably you would know the headers (you will certainly know the struct fields if you’re using .structs_from_csv). If not, you can do
|> Stream.map(&inspect/1) \
or
|> Stream.map(&(&1 |> Map.from_struct |> Enum.reduce("", fn({k, v}, acc) -> "#{acc}#{k}: #{v}, " end) |> String.slice(0..-3))) \
(for cleaner formatting) instead.
You could try inspect(limit: :infinity, pretty: true, width: 60) (mess with width).
Yes.
Does this line mean a function which appends a newline to the input, assuming that &1 refers to the first parameter?
The Enum.to_list function starts the printing of each new struct on a new line, but inspect(limit: :infinity) \ appears to remove them before they get to the (&File.write("output.txt", &1)).().
Is there an alternative that will maintain the line breaks?
Can the output of Enum.to_list be written directly to a file without the inspect removing the line breaks?
The structs are separated at },. How would a search and replace to replace them with },\n be written (assumes that \n is the right code for a line break?
How would I place the output of inspect(limit: :infinity) \ into a variable and run the search and replace for }, with },\n before the file write?
You can JSON’ify it and send it through a formatter.
Or you can parse out each list individually, maybe by replacing your inspect line with this one instead:
|> (fn l ->[?[, Enum.map(l, &([inspect(limit: :infinity), ?\n])), ?]] end).()
Or something like that (untested code), or grab a formatting library. ^.^
I finally got the main logic of the working with this code.
File.stream!('tmp.csv') \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> Enum.map(fn(x) -> inspect(x, [limit: :infinity])<>",\n" end) \
|> (&File.write('output.txt',&1)).()
Each line is appended with a comma. What I want to do now is to add an opening bracket for the list [, ignore the comma on the last item , and add the closing bracket `]’.
1. if it is the first item prepend the '['.
2. output the item
3. if is not the last item append a `,`
4. if all have been added, append the closing `]`.
How can I do that in snazzy one-liner. In the same pipelined set of functions. I want to do it in a more functional style , not I would normally do it in procedural languages.
The normal way is to ascertain the size of the list, set a variable to the no of items processed, if 0 items, prepend the [, print the item, increment the printed count, if not same as no of items, append the the , , of the last tiem append the closing bracket. This approach can be prone to off by one errors, which is why I want to try a more functional style.
I am beginning to get the hang of your formula after realizing that the ?[ and ?\n refer to unicode code points.
File.stream!('tmp.csv') \
|> DataMorph.structs_from_csv("open-register", :iso_country) \
|> (fn l ->[?[, Enum.map(l, &([inspect(limit: :infinity), ?\n])), ?]] end).()
|> (&File.write('output.txt',&1)).()
When I apply your formula this is the message error:
* (CompileError) iex:55: invalid args for &, expected an expression in the format of &Mod.fun/arity, &local/arity or a capture containing at least one argument as &1, got: [inspect(limit: :infinity), 10]
(elixir) expanding macro: Kernel.|>/2
iex:55: (file)
It is a syntax error which doesn’t get to compile.
Uh sorry, like I said untested code. ^.^;
The: inspect(limit: :infinity)
Should be: inspect(&1, limit: :infinity)
© Copyright Elixir Forum | Terms | Privacy & Cookies
