thanos
ExArrow - Apache Arrow IPC, Flight & ADBC Support
ExArrow – Apache Arrow IPC, Flight & ADBC Support for Elixir
I’m announcing the public release of ExArrow:
ExArrow provides a focused, BEAM-friendly implementation of:
- Arrow IPC encoding / decoding
- Arrow Flight client support
- Arrow Database Connectivity (ADBC) integration
- Explorer interoperability
The library concentrates on Arrow’s memory model and transport protocols.
Design Scope
ExArrow is not:
- A dataframe library
- A database driver
- A replacement for Explorer or ADBC
It is the Arrow-native infrastructure layer.
It exposes Arrow in a way that integrates cleanly with existing Elixir data tooling.
Ecosystem Positioning
| Layer | Library |
|---|---|
| Dataframe computation | Explorer |
| Database connectivity | ADBC |
| Chunked storage | ExZarr |
| Columnar transport & memory | ExArrow |
ExArrow complements these libraries by handling Arrow IPC, Flight, and columnar structures directly.
How It Fits Into Real Workflows
Typical architecture:
- ADBC retrieves Arrow data from a database
- ExArrow handles IPC or Flight transport
- Explorer consumes Arrow-backed dataframes
- ExZarr stores large chunked datasets when persistence is required
This layered model enables:
- Zero-copy data interchange
- Clean separation of responsibilities
- Cross-language interoperability
- Arrow-native services in Phoenix or distributed BEAM systems
Practical Impact
Apache Arrow has become a core interoperability layer in modern analytics systems.
ExArrow makes Arrow directly usable in Elixir without requiring external bridges or conversion layers.
The intent is to provide a stable foundation for columnar-native applications on the BEAM.
Contributions, benchmarks, and production feedback are welcome.
— Thanos Vassilakis
First Post!
filmor
Could you please check the docs at resource_impl in rustler - Rust and implement the resources accordingly? The way you are doing it right now hooks into internal types and is needlessly complicated.
Last Post!
Popular in Announcing
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security









