imed-code
Hello,
I’m trying to load a json API message with Explorer.DataFrame.
My api is returning a list like bellow:
resp = [%{"idade" => "8", "nome" => "Mirela"}, %{"idade" => "40", "nome" => "Sergio"}]
When I try to run the following code:
require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(resp), lazy: true)
I got an error:
** (ErlangError) Erlang error: :nif_panicked
(explorer 0.5.7) Explorer.PolarsBackend.Native.df_load_ndjson("...
But if I change to read the first element it works as well.
require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(List.first(resp)))
How to read the entire data ?
Thank you.
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
**Building the future of software with AI, trust, and human connection.**
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First Post!- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
al2o3cr
The ndjson parser expects JSON objects separated by newlines - for instance, here’s an example from Explorer’s tests:
https://github.com/elixir-nx/explorer/blob/1db7146d83965f6446c4ab52816be4993952724c/test/explorer/data_frame/ndjson_test.exs#L36-L47
That is not what
Jason.encode!(resp)will produce - that result will have[at the beginning and,between the objects. You’d need to encode each item inrespseparately.HOWEVER
In general, re-encoding like that should make you suspect that you’re not using the best API for your situation.
The better choice is
Explorer.DataFrame.new, in particular the last example labeled “From row data:”.