Just got into ResourceArc with Rustler and it’s wild. I learned it from Discord who use this in their systems.
Allows you to hold references to mutable objects inside your Elixir functions and states and perform transformative Rust operations on them, get data and out.
Once you have this running Elixir really starts feeling a lot like any other programming language. The only remaining quirk after that is still dealing with the weird syntax of GenServers and Supervisors.
If UI element and mouse/keyboard/touchscreen input packages were more advanced it would be quite easy to build full desktop applications in Elixir no different than any other language.
It was nice of Discord to share their project. Would have been hard to figure out the syntax otherwise. ChatGPT hallucinates on Elixir as always. Few to no comprehensible examples I could find.
I have not see this yet as a problem personally, but I note here:
How the Erlang VM runs code in Erlang processes is a bit strange. You have a small amount OS threads which acts as Erlang schedulers (by default there is one thread per CPU core). These scheduler threads are what actually runs your Erlang code. The special part about how Erlang does scheduling, is that it doesn’t allow a single process to run for more then a small amount of time before it reschedules it and starts executing another process. What this means for the user is that a single process which takes up a lot of CPU time, will not degrade the latency of any of the other (millions?) of processes running within the VM. If you are interested in knowing more about Erlang’s schedulers, this is a good article that goes into further details.
Since NIFs run within the scheduler threads by default, bad things will happen to the latency and performance of processes within the VM if a NIF runs for more than around a millisecond. This should be avoided, and care therefore needs to be taken to avoid blocking the scheduler thread for too long.
Luckily, a few techniques can be used to avoid this.
- Do the computation in smaller chunks, making sure to return from the NIF before too much time is taken.
- Put the computation on a separate thread, potentially in a thread pool. When the computation is finished, send the result back to the correct Erlang process.
The entire point of the mutable state of ResourceArc is that you are likely generally going to use it for computationally expensive data transformations, like Discord, or worse.
So let’s say you have a hypothetical:
{:ok, rust_resource_ref} = RustModule.create_resource_arc()
RustModule.do_heavy_work(rust_resource_ref) # tough mutable object work to do
Is there any easy way to do what they suggest? ie. #2? Put the “heavy work” on another thread and have it send a message to a GenServer or other PID when done as an alert? What might that look like? Ideally with no third party solution?
Thanks for any thoughts.
