sdr
I’ve been playing around with Rustler NIFs to get familiar with them and understand how they interact with Elixir.
While running some simple toy NIFS and benchmarking them to be impressed with the performance differences, I was surprised with :process-memory-high-watermark warnings and opening the observer showed that memory had ballooned up to a couple of GBs.
The code isn’t creating any long term data and simply sums the numbers from A..B so I was surprised to see the jump in memory. The create is built release mode and benched with Benchee and it’s unclear what’s causing the memory jump.
Code is as follows
#[rustler::nif]
fn add(a: i64, b: i64) -> i64 {
let mut r:i64 = 0;
for x in a..b {
r += x;
}
return r;
}
rustler::init!("Elixir.FastMath", [add]);
Any hints on where to look or insights into what’s causing the high memory usage?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Master modern tools, write cleaner code, and tackle real-world challenges with Manning books, videos, and courses. Start building today.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #ai
- #elixirconf-us
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Master modern tools, write cleaner code, and tackle real-world challenges.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 6- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
sdr
A minimal repo to replicate this can be found at GitHub - sross/niftest · GitHub
dimitarvp
I wonder if you tried with a more recent Rustler version btw?
If this is still true it would be quite worrying.
bdarla
I experimented with the provided repo and here are the results in my machine (Intel MBP with latest versions of Elixir and Rust as well as the employed libraries, i.e. rustler v0.26 and benchee v1.1).
I am not sure whether the issue remains or not. If anyone could judge better than me, please, do so.
dimitarvp
Yikes, north of 400MB is not good at all.
sdr
Can confirm that I’m still seeing the same behaviour with rustler 0.26.0 and benchee 1.1.0 on Erlang 1.14 with OTP 25 under WSL.
Memory spikes to just under 3GB and the drops back to 1.2GB. the drop back to normal levels at the 6 second mark is when I ran a manual garbage collect.
sdr
After building some elixir/erlang muscle this has nothing to do with NIFs but is caused by Benchee collecting a list of samples for each iteration.
The same memory usage can be seen with a no-op function:
Benchee.run(%{fast: fn -> :ok end})For short function calls this results in a large list (order of millions) of samples which explains the the memory spike.