imed-code

imed-code

Explorer.DataFrame - load_ndjson json

Hello,

I’m trying to load a json API message with Explorer.DataFrame.

My api is returning a list like bellow:

resp = [%{"idade" => "8", "nome" => "Mirela"}, %{"idade" => "40", "nome" => "Sergio"}]

When I try to run the following code:

require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(resp), lazy: true)

I got an error:

** (ErlangError) Erlang error: :nif_panicked
    (explorer 0.5.7) Explorer.PolarsBackend.Native.df_load_ndjson("...

But if I change to read the first element it works as well.

require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(List.first(resp)))

How to read the entire data ?

Thank you.

First Post!

al2o3cr

al2o3cr

The ndjson parser expects JSON objects separated by newlines - for instance, here’s an example from Explorer’s tests:

https://github.com/elixir-nx/explorer/blob/1db7146d83965f6446c4ab52816be4993952724c/test/explorer/data_frame/ndjson_test.exs#L36-L47

That is not what Jason.encode!(resp) will produce - that result will have [ at the beginning and , between the objects. You’d need to encode each item in resp separately.

HOWEVER

In general, re-encoding like that should make you suspect that you’re not using the best API for your situation.

The better choice is Explorer.DataFrame.new, in particular the last example labeled “From row data:”.

Where Next?

Popular in Questions Top

RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement