After 3 days of pain, I successfully added rustler_precompiled to kuzu_nif and published to hex.pm. Users of kuzu_nif on supported targets (see below) no longer need to compile the Rust crate.

Why? This saves us a few minutes on every build. For me, I’m using kuzu_nif in a Phoenix Framework project. Having a precompiled version of kuzu_nif means 1) I don’t need to install a Rust compiler in my Phoenix Framework dockerfile and 2) I save precious time compiling my Phoenix app.

• hex.pm/packages/kuzu_nif
• GitHub: Release v0.7.0

The supported targets are indicated in the packaged release files. TLDR; Apple arm/x86 and Debian Linux arm/x64.

• libkuzu_ex-v0.7.0-nif-2.17-aarch64-apple-darwin.so.tar.gz
• libkuzu_ex-v0.7.0-nif-2.17-aarch64-unknown-linux-gnu.so.tar.gz
• libkuzu_ex-v0.7.0-nif-2.17-x86_64-apple-darwin.so.tar.gz
• libkuzu_ex-v0.7.0-nif-2.17-x86_64-unknown-linux-gnu.so.tar.gz

Figuring out how to successfully compile on aarch64 linux was the hardest part. In the end, I had to compile a custom Docker image just for building aarch64-unknown-linux-gnu.

Enjoy!

The thread creation stuff in enif only exists to provide a platform-independent API for threading in C. They don’t have (to my knowledge) any deeper “integration” with the BEAM.

One thing to keep in mind is that not all work has to happen in the actual NIF call. You can easily spawn a thread (or a pool of threads) and pass the PID of the caller that you then use to send the result of the query back to. This is what I use in my XML processing faster_xml/rust_src/faster_xml_nif/src/lib.rs at master · filmor/faster_xml · GitHub and it works fine in production, even without an actual thread pool.

So, simple suggestion to get started: Have a Database stored in a resource object, and on every query, spawn a thread that creates a Connection and sends the result back.

Later you can refine this to use a thread pool (with some retry logic on the Elixir side).

That way, the NIF will be quick enough that you don’t need to worry about scheduling and you get proper concurrent processing.

NIFs are executed by regular scheduled threads of the VM. See [the Erlang docs] (erl_nif — erts v15.1.2) for the big fat warning on long-running work, special care should be taken when a call takes more time than about 1ms. I usually just scheduled NIFs on an async scheduler (dirty CPU mostly) when the NIF was supposed to work for some longer time.

Note that Rustler doesn’t implement all of erl_nif, for example creating threads in the VM is not implemented as rust threads can be spawned if necessary.