High memory usage when performance testing simple Rustler NIFs

I’ve been playing around with Rustler NIFs to get familiar with them and understand how they interact with Elixir.

While running some simple toy NIFS and benchmarking them to be impressed with the performance differences, I was surprised with :process-memory-high-watermark warnings and opening the observer showed that memory had ballooned up to a couple of GBs.

The code isn’t creating any long term data and simply sums the numbers from A…B so I was surprised to see the jump in memory. The create is built release mode and benched with Benchee and it’s unclear what’s causing the memory jump.

Code is as follows

#[rustler::nif]
fn add(a: i64, b: i64) -> i64 {
    let mut r:i64 = 0;
    for x in a..b {
        r += x;
    }
    return r;
}

rustler::init!("Elixir.FastMath", [add]);

Any hints on where to look or insights into what’s causing the high memory usage?

1 Like

A minimal repo to replicate this can be found at https://github.com/sross/niftest

2 Likes

I wonder if you tried with a more recent Rustler version btw?

If this is still true it would be quite worrying.

I experimented with the provided repo and here are the results in my machine (Intel MBP with latest versions of Elixir and Rust as well as the employed libraries, i.e. rustler v0.26 and benchee v1.1).
I am not sure whether the issue remains or not. If anyone could judge better than me, please, do so.

1 Like

Yikes, north of 400MB is not good at all. :grimacing: :103:

Can confirm that I’m still seeing the same behaviour with rustler 0.26.0 and benchee 1.1.0 on Erlang 1.14 with OTP 25 under WSL.

Memory spikes to just under 3GB and the drops back to 1.2GB. the drop back to normal levels at the 6 second mark is when I ran a manual garbage collect.

2 Likes