The Functionality section in the erl_nif docs mention 3 different strategies for long running NIFs: Yielding NIFs, Threaded NIFs and Dirty NIFs. The rustler docs clearly mention how to implement Dirty NIFs. I also found an obscure function rustler::thread::spawn that handles setting up a non-Erlang thread to execute a closure and returning the result to the calling Erlang process. This addresses Threaded NIFs.

However, I can’t find a Rust-like way to write Yielding NIFs i.e. NIFs that perform work in chunks, yielding back control to the BEAM every millisecond. I found Rustler’s bindings to the C NIF API, so I could just call enif_schedule_nif itself and be done with it. Unfortunately, I can foresee two problems with this approach:

• Calling the C API kind of defeats the purpose of writing in Rust
• Calling the C API from Rust is probably far more painful than writing in C in the first place

TLDR; How do you implement Yielding NIFs in Rustler?

In order to eliminate any chance of an XY problem, here’s some more context: I am writing an NIF for a heavily async Rust library. I’ve also chosen tokio as the async runtime. I reckon that since I am binding to asynchronous functions, I might as well pass on the benefits to the BEAM’s scheduler.

A yielding NIF seemed like a better fit than a threaded NIF to me for 2 reasons:

• Tokio provides robust scheduling and threading, and exposes this in a task-oriented API, making rustler::thread::spawn and enif_thread_* seem redundant
• Yielding NIFs are recommended over Threaded and Dirty NIFs by the docs

Since Tokio threads are separate from the BEAM’s, I can have an initial NIF call create a task, and following calls yield very quickly until a result is received.