thanos
ExArrow - Apache Arrow IPC, Flight & ADBC Support
ExArrow – Apache Arrow IPC, Flight & ADBC Support for Elixir
I’m announcing the public release of ExArrow:
ExArrow provides a focused, BEAM-friendly implementation of:
- Arrow IPC encoding / decoding
- Arrow Flight client support
- Arrow Database Connectivity (ADBC) integration
- Explorer interoperability
The library concentrates on Arrow’s memory model and transport protocols.
Design Scope
ExArrow is not:
- A dataframe library
- A database driver
- A replacement for Explorer or ADBC
It is the Arrow-native infrastructure layer.
It exposes Arrow in a way that integrates cleanly with existing Elixir data tooling.
Ecosystem Positioning
| Layer | Library |
|---|---|
| Dataframe computation | Explorer |
| Database connectivity | ADBC |
| Chunked storage | ExZarr |
| Columnar transport & memory | ExArrow |
ExArrow complements these libraries by handling Arrow IPC, Flight, and columnar structures directly.
How It Fits Into Real Workflows
Typical architecture:
- ADBC retrieves Arrow data from a database
- ExArrow handles IPC or Flight transport
- Explorer consumes Arrow-backed dataframes
- ExZarr stores large chunked datasets when persistence is required
This layered model enables:
- Zero-copy data interchange
- Clean separation of responsibilities
- Cross-language interoperability
- Arrow-native services in Phoenix or distributed BEAM systems
Practical Impact
Apache Arrow has become a core interoperability layer in modern analytics systems.
ExArrow makes Arrow directly usable in Elixir without requiring external bridges or conversion layers.
The intent is to provide a stable foundation for columnar-native applications on the BEAM.
Contributions, benchmarks, and production feedback are welcome.
— Thanos Vassilakis
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Phoenix, LiveView, multimedia pipelines, distributed systems. 400 engineers, Elixir since 2017. Available for hire.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #elixirconf-us
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #performance
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 3 of 3 Posts!
filmor
Could you please check the docs at resource_impl in rustler - Rust and implement the resources accordingly? The way you are doing it right now hooks into internal types and is needlessly complicated.
thanos
Thanks - will do - I’ll try and fix it in the next release, meanwhile… …I’ve release 0.3.0 - No breaking changes.
-–
thanos
Add an issue The way you are doing it right now hooks into internal types and is needlessly complicated. · Issue #106 · thanos/ex_arrow · GitHub