alphydan

alphydan

Error when importing CSV with Explorer.DataFrame.from_csv! possibly related to dtype

Hello,

I’m an elixir beginner, exploring livebook. I’m running a local notebook on a linux machine. Initially I tried to import a CSV file with Explorer like so:

alias Explorer.{DataFrame, Series}
path_to_csv = "./some_file.csv"
DataFrame.from_csv!(path_to_csv)

However, I am seeing the following error:

** (RuntimeError) from_csv failed: {:polars, "Could not parse `Fee Amount` 
    as dtype Int64 at column 16.\nThe current offset in the file is 249 bytes.\n\n
    Consider specifying the correct dtype, increasing\n
    the number of records used to infer the schema,\n
    enabling the `ignore_errors` flag, or adding\n`Fee Amount` to the `null_values` list."}
    (explorer 0.5.5) lib/explorer/data_frame.ex:438: Explorer.DataFrame.from_csv!/2

To check if the problem was indeed with the data in the column Fee Amount, I filled the column with integers. But I get the same error. What can I try to better diagnose this problem?

Most Liked

BradS2S

BradS2S

Oh yes the error message could be more helpful but it is a parsing error. It sees the newline character as a new row. It looks forward and sees mostly numbers in the row so it decides to infer int64 which doesn’t work for the second half of the string.

LostKobrakai

LostKobrakai

I’m totally fine with headers be required to be single line. I’m just argueing that this should not be a CSV parsing error, but a proper “hey we need you to have single line headers” error.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hi @alphydan welcome! Are you able to provide a sample of that CSV file?

Where Next?

Popular in Questions Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "eq...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41454 115
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30840 112
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement