Semurg - a CPU-native data engine built with Elixir and Rust

Hi all, long-time reader, finally made an account.

I wanted to introduce something my team and I have been building, mostly because the core of it is an Elixir and Rust bet and I would really like this community’s take on it.

Semurg holds every type of data (relational, graph, object, document, search, vector, time-series) as one fixed 64-byte binary container, in a single engine over one copy of the data, CPU-only, on hardware you control. It is a commercial product and closed source. It is also live with no login at semurg.io so you are very welcome to have a play with it.

The part that belongs on this forum is how it is put together:

BEAM conducts, Rust does the heavy lifting. Elixir and OTP own supervision, scheduling and fault tolerance. The per-byte work runs in Rust NIFs, one concurrent arm per physical core, with no shared write lock between them. We hand each NIF a whole batch and fan out inside a single crossing rather than paying the boundary cost per item, and only tokens and offsets ever cross, never materialised data. Long jobs go on dirty schedulers, and every NIF has a pure-BEAM fallback, so a missing or failed native artifact costs us throughput rather than taking the VM down.

The 64 bytes are not arbitrary. One container is a CPU cache line, 64 of them is a 4KB page, so the storage format is aligned to what the hardware actually moves.

Agents are just supervised BEAM processes over the same store, so a swarm gets isolation and concurrency from the runtime rather than from containers.

On scope, so I am not overselling it: the substrate and a gateway that tokenises PII before any external model call are live today. Growing CPU-native models on the same substrate is still research, and I try to keep that line clear.

There is an independent benchmark running at a university lab at the moment which I will happily share when it lands.

Happy to go into as much depth as anyone wants on the orchestration and the BEAM side. The storage layout and IO mechanics are the bit we keep closed, so I will be vaguer there, but everything above that is fair game.

What I would most like is to hear from anyone who has run BEAM and Rust NIFs in production, especially dirty schedulers and large batched crossings. What caught you out? I have a feeling we are not the first to learn some of these the hard way.

Thanks,
Slade