I just wanted to post about my remarkable experience with Rustler today.
Unbelievable good.
Note: this example is an updated version of an article I found online. Main difference is being able to return binary data from Rust, and get working on a MacBook M1.
Short version to get up and running:
mix new base64
cd base64
edit mix.exs, add {:rustler, "~> 0.22.2"}
mix deps.get
mix rustler.new
In my case, Elixir module name was Base64, Rust crate name was base64_nif
- note: I used
base64_nifto avoid name collision with rust cratebase64
Add base64 = "0.13.0" at the end of native/base64_nif/Cargo.toml
Edit native/base64_nif/src/lib.rs as follows:
use base64;
use rustler::Binary;
use rustler::OwnedBinary;
#[rustler::nif]
pub fn encode(binary: Binary) -> String {
base64::encode(binary.as_slice())
}
#[rustler::nif]
pub fn decode(b64: Binary) -> OwnedBinary {
let bytes = base64::decode(b64.as_slice()).expect("decode failed: invalid base64");
let mut binary: OwnedBinary = OwnedBinary::new(bytes.len()).unwrap();
binary.as_mut_slice().copy_from_slice(&bytes);
return binary
}
rustler::init!("Elixir.Base64", [encode, decode]);
Note on the code above! It took me a while to figure out how to return a
binaryfrom Rust. This does not seem to be well documented.
Edit lib/base64.ex as follows:
defmodule Base64 do
use Rustler, otp_app: :base64, crate: "base64_nif"
@spec decode(String.t()) :: binary
def decode(_base64), do: :erlang.nif_error(:nif_not_loaded)
@spec encode(binary) :: binary
def encode(_binary), do: :erlang.nif_error(:nif_not_loaded)
end
If you are on a Macbook M1
Add this to native/base64_nif/.cargo/config:
[target.aarch64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
(this is fixed in github, but not pushed to hex yet as of 0.22.2.
That’s it! Amazing!
Results for encoding small data:
iex(3)> file = "this is a really small string that we are going to encode to base 64"
"this is a really small string that we are going to encode to base 64"
iex(4)> Benchee.run(
...(4)> %{
...(4)> "elixir_base64" => fn -> Base.encode64(file) end,
...(4)> "rust_base64" => fn -> Base64.encode(file) end
...(4)> },
...(4)> time: 2,
...(4)> memory_time: 2
...(4)> )
...
Benchmarking elixir_base64...
Benchmarking rust_base64...
Name ips average deviation median 99th %
rust_base64 7.75 M 129.10 ns ±9810.57% 0 ns 0 ns
elixir_base64 1.70 M 587.28 ns ±1714.81% 0 ns 2000 ns
Comparison:
rust_base64 7.75 M
elixir_base64 1.70 M - 4.55x slower +458.18 ns
Memory usage statistics:
Name Memory usage
rust_base64 560 B
elixir_base64 784 B - 1.40x memory usage +224 B
Rust is 4.55x faster
Results on larger binary data:
Benchmarking elixir_base64...
Benchmarking rust_base64...
Name ips average deviation median 99th %
rust_base64 2.33 K 0.43 ms ±3.23% 0.43 ms 0.47 ms
elixir_base64 0.100 K 9.97 ms ±0.67% 9.95 ms 10.18 ms
Comparison:
rust_base64 2.33 K
elixir_base64 0.100 K - 23.19x slower +9.54 ms
Memory usage statistics:
Name Memory usage
rust_base64 560 B
elixir_base64 784 B - 1.40x memory usage +224 B
Rust is 23.19x faster
ps don’t forget to run with MIX_ENV=prod iex -S mix otherwise elixir will be 2x faster!
Wow. Thanks so much to the folks that worked on rustler!
It’s a game changer!
