alphydan

alphydan

Error when importing CSV with Explorer.DataFrame.from_csv! possibly related to dtype

Hello,

I’m an elixir beginner, exploring livebook. I’m running a local notebook on a linux machine. Initially I tried to import a CSV file with Explorer like so:

alias Explorer.{DataFrame, Series}
path_to_csv = "./some_file.csv"
DataFrame.from_csv!(path_to_csv)

However, I am seeing the following error:

** (RuntimeError) from_csv failed: {:polars, "Could not parse `Fee Amount` 
    as dtype Int64 at column 16.\nThe current offset in the file is 249 bytes.\n\n
    Consider specifying the correct dtype, increasing\n
    the number of records used to infer the schema,\n
    enabling the `ignore_errors` flag, or adding\n`Fee Amount` to the `null_values` list."}
    (explorer 0.5.5) lib/explorer/data_frame.ex:438: Explorer.DataFrame.from_csv!/2

To check if the problem was indeed with the data in the column Fee Amount, I filled the column with integers. But I get the same error. What can I try to better diagnose this problem?

Most Liked

BradS2S

BradS2S

Oh yes the error message could be more helpful but it is a parsing error. It sees the newline character as a new row. It looks forward and sees mostly numbers in the row so it decides to infer int64 which doesn’t work for the second half of the string.

LostKobrakai

LostKobrakai

I’m totally fine with headers be required to be single line. I’m just argueing that this should not be a CSV parsing error, but a proper “hey we need you to have single line headers” error.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hi @alphydan welcome! Are you able to provide a sample of that CSV file?

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
earth10
Hi, I’m just starting to build a side-project with Elixir and Phoenix and doing some basic test with Elixir alone. What strikes me is th...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement