imed-code
Hello,
I’m trying to load a json API message with Explorer.DataFrame.
My api is returning a list like bellow:
resp = [%{"idade" => "8", "nome" => "Mirela"}, %{"idade" => "40", "nome" => "Sergio"}]
When I try to run the following code:
require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(resp), lazy: true)
I got an error:
** (ErlangError) Erlang error: :nif_panicked
(explorer 0.5.7) Explorer.PolarsBackend.Native.df_load_ndjson("...
But if I change to read the first element it works as well.
require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(List.first(resp)))
How to read the entire data ?
Thank you.
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Error tracking, logs, APM, uptime and cron monitoring in one dev-friendly platform. Find and fix problems before your users notice.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #ai
- #iex
- #graphql
- #elixirconf-us
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 1- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
al2o3cr
The ndjson parser expects JSON objects separated by newlines - for instance, here’s an example from Explorer’s tests:
https://github.com/elixir-nx/explorer/blob/1db7146d83965f6446c4ab52816be4993952724c/test/explorer/data_frame/ndjson_test.exs#L36-L47
That is not what
Jason.encode!(resp)will produce - that result will have[at the beginning and,between the objects. You’d need to encode each item inrespseparately.HOWEVER
In general, re-encoding like that should make you suspect that you’re not using the best API for your situation.
The better choice is
Explorer.DataFrame.new, in particular the last example labeled “From row data:”.