ExArrow – Apache Arrow IPC, Flight & ADBC Support for Elixir
I’m announcing the public release of ExArrow:
ExArrow provides a focused, BEAM-friendly implementation of:
- Arrow IPC encoding / decoding
- Arrow Flight client support
- Arrow Database Connectivity (ADBC) integration
- Explorer interoperability
The library concentrates on Arrow’s memory model and transport protocols.
Design Scope
ExArrow is not:
- A dataframe library
- A database driver
- A replacement for Explorer or ADBC
It is the Arrow-native infrastructure layer.
It exposes Arrow in a way that integrates cleanly with existing Elixir data tooling.
Ecosystem Positioning
| Layer | Library |
|---|---|
| Dataframe computation | Explorer |
| Database connectivity | ADBC |
| Chunked storage | ExZarr |
| Columnar transport & memory | ExArrow |
ExArrow complements these libraries by handling Arrow IPC, Flight, and columnar structures directly.
How It Fits Into Real Workflows
Typical architecture:
- ADBC retrieves Arrow data from a database
- ExArrow handles IPC or Flight transport
- Explorer consumes Arrow-backed dataframes
- ExZarr stores large chunked datasets when persistence is required
This layered model enables:
- Zero-copy data interchange
- Clean separation of responsibilities
- Cross-language interoperability
- Arrow-native services in Phoenix or distributed BEAM systems
Practical Impact
Apache Arrow has become a core interoperability layer in modern analytics systems.
ExArrow makes Arrow directly usable in Elixir without requiring external bridges or conversion layers.
The intent is to provide a stable foundation for columnar-native applications on the BEAM.
Contributions, benchmarks, and production feedback are welcome.
— Thanos Vassilakis
