alphydan
Error when importing CSV with Explorer.DataFrame.from_csv! possibly related to dtype
Hello,
I’m an elixir beginner, exploring livebook. I’m running a local notebook on a linux machine. Initially I tried to import a CSV file with Explorer like so:
alias Explorer.{DataFrame, Series}
path_to_csv = "./some_file.csv"
DataFrame.from_csv!(path_to_csv)
However, I am seeing the following error:
** (RuntimeError) from_csv failed: {:polars, "Could not parse `Fee Amount`
as dtype Int64 at column 16.\nThe current offset in the file is 249 bytes.\n\n
Consider specifying the correct dtype, increasing\n
the number of records used to infer the schema,\n
enabling the `ignore_errors` flag, or adding\n`Fee Amount` to the `null_values` list."}
(explorer 0.5.5) lib/explorer/data_frame.ex:438: Explorer.DataFrame.from_csv!/2
To check if the problem was indeed with the data in the column Fee Amount, I filled the column with integers. But I get the same error. What can I try to better diagnose this problem?
Most Liked
BradS2S
Oh yes the error message could be more helpful but it is a parsing error. It sees the newline character as a new row. It looks forward and sees mostly numbers in the row so it decides to infer int64 which doesn’t work for the second half of the string.
LostKobrakai
I’m totally fine with headers be required to be single line. I’m just argueing that this should not be a CSV parsing error, but a proper “hey we need you to have single line headers” error.







