Using Rustler with OTP24

Compiling any app dependent on Rustler is not working on Elixir v12.0 and OTP-24

Anyone having similar issues?

Could not compile :rustler, no "mix.exs", "rebar.config" or "Makefile" (pass :compile as an option to customize compilation, set it to "false" to do nothing)
Unchecked dependencies for environment prod:
* rustler (https://github.com/rusterlium/rustler.git)
  could not find an app file at "c:/Projects/bat/_build/prod/lib/rustler/ebin/rustler.app". This may happen if the dependency was not yet compiled or the dependency indeed has no app file (then you can pass app: false as option)
could not compile dependency :xler, "mix compile" failed. You can recompile this dependency with "mix deps.compile xler", update it with "mix deps.update xler" or clean it with "mix deps.clean xler"

Is that v1.12.0 or one of the RCs? We fixed a bug on the RCs but it should be fixed on the final release.

v1.12.0 sir

Looking at the Rustler project, it indeed does not have any of mix.exs/rebar.config/makefile at the root, so it makes sense that Elixir does not know how to compile it: GitHub - rusterlium/rustler: Safe Rust bridge for creating Erlang NIF functions

Try using a :sparse checkout on the relevant subdirectory or use the Hex package. I am not sure how it worked before (it shouldn’t have).

Which version of rustler are you using? Mine is 0.22.0-rc.0. My Elixir+Rust project compiles fine with Erlang 23.3.4 and Elixir 1.12.0-otp-23, but fails to compile with Erlang 24.0 and Elixir 1.12.0-otp-24. The error is:

Caused by:
  process didn't exit successfully: `/Users/dimi/kod/xqlite/_build/dev/rustler_crates/xqlitenif/release/build/rustler-69fb4c768ce25323/build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at 'Erlang version 2.16 not handled, please file a a bug report.', /Users/dimi/.cargo/registry/src/github.com-1ecc6299db9ec823/rustler-0.22.0-rc.0/build.rs:36:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
** (RuntimeError) Rust NIF compile error (rustc exit code 101)
    (rustler 0.22.0-rc.0) lib/mix/tasks/compile.rustler.ex:89: Mix.Tasks.Compile.Rustler.compile_crate/1
    (elixir 1.11.4) lib/enum.ex:1411: Enum."-map/2-lists^map/1-0-"/2
    (rustler 0.22.0-rc.0) lib/mix/tasks/compile.rustler.ex:28: Mix.Tasks.Compile.Rustler.run/1
    (mix 1.11.4) lib/mix/task.ex:394: Mix.Task.run_task/3
    (mix 1.11.4) lib/mix/tasks/compile.all.ex:90: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.11.4) lib/mix/tasks/compile.all.ex:70: Mix.Tasks.Compile.All.compile/4
    (mix 1.11.4) lib/mix/tasks/compile.all.ex:57: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.11.4) lib/mix/tasks/compile.all.ex:35: Mix.Tasks.Compile.All.run/1

Seems there’s a hard requirement for a maximum Erlang version. I thought of contributing and maybe even taking over rustler in the future (the owners are semi-open to the idea btw) but life and work keep getting in the way. :frowning:

Still, maybe I can muster a small PR that fixes this problem soon-ish.


TL;DR: It’s not Elixir 1.12.0’s fault. It seems to be rustler insisting on a maximum Erlang version.

2 Likes

This has been updated on the main branch but not yet published as a release. if you use the git repo as your dep, you won’t get that error

I am now using Erlang 24.0 and Elixir 1.12.0-otp-24 plus I use rustler directly from its GitHub repo and I am still getting the error. Which versions do you have exactly so it’s working for you?

1 Like

I still have problems, not fully resolved. but at least The main branch of rustler is updated for OTP-24

Could not compile :rustler, no "mix.exs", "rebar.config" or "Makefile" (pass :compile as an option to customize compilation, set it to "false" to do nothing)
==> meeseeks_html5ever
Could not compile :rustler, no "mix.exs", "rebar.config" or "Makefile" (pass :compile as an option to customize compilation, set it to "false" to do nothing)
Unchecked dependencies for environment prod:
* rustler (https://github.com/rusterlium/rustler.git)
  the dependency build is outdated, please run "MIX_ENV=prod mix deps.compile"
could not compile dependency :meeseeks_html5ever, "mix compile" failed. You can recompile this dependency with "mix deps.compile meeseeks_html5ever", update it with "mix deps.update meeseeks_html5ever" or clean it with "mix deps.clean meeseeks_html5ever"
2 Likes

@josevalim or @AstonJ Can you split this mini-thread off of the main one? We kind of got off-topic.

1 Like

I had the same problem - it all seems to work after you use the git checkout option both in your mix.exs as well as Cargo.toml (a short blog post about this here).

I was missing the Cargo.toml step for a long time leading me to being stuck with 0.22.0-rc.0 there and still getting the version error.

To summarize, I’ve added this to mix.exs (thanks for the sparse tip @josevalim):

{:rustler, git: "git://github.com/rusterlium/rustler.git", branch: "master", sparse: "rustler_mix"},

And this to Cargo.toml:

rustler = {git = "git://github.com/rusterlium/rustler.git", branch="master"}
2 Likes

no luck compiling {:meeseeks, "~> 0.15.1"}, even with the specified updates.

1 Like

There’s a 0.22.0-rc.1 release that got out like 45 minutes ago.

@maciejgryka What you have shown have worked for me for Elixir 1.12.0-otp-24 and Erlang 24.0.1. Thanks!

(I still used rustler from the master branch, though.)

4 Likes

Still have the same issues after this update. Previously working code will not compile anymore.


   Compiling quote v1.0.7
   Compiling rustler_codegen v0.22.0-rc.1
error[E0432]: unresolved imports `rustler_sys::TWinDynNifCallbacks`, `rustler_sys::WIN_DYN_NIF_CALLBACKS`
  --> C:\Users\Charles Okwuagwu\.cargo\registry\src\github.com-1ecc6299db9ec823\rustler-0.22.0-rc.1\src\codegen_runtime.rs:16:23
   |
16 | pub use rustler_sys::{TWinDynNifCallbacks, WIN_DYN_NIF_CALLBACKS};
   |                       ^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^ no `WIN_DYN_NIF_CALLBACKS` in the root
   |                       |
   |                       no `TWinDynNifCallbacks` in the root

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `rustler`

To learn more, run the command again with --verbose.

== Compilation error in file lib/nif.ex ==
** (RuntimeError) Rust NIF compile error (rustc exit code 101)
    (rustler 0.22.0-rc.1) lib/rustler/compiler.ex:36: Rustler.Compiler.compile_crate/2
    lib/nif.ex:2: (module)
    (stdlib 3.15) erl_eval.erl:685: :erl_eval.do_apply/6

Show us your mix.exs?

defp deps do

    [

      {:tzdata, "~> 1.1.0"},

      {:gen_smtp, "~> 1.0"},

      {:mime, "~> 1.4"},

      {:fs, "~> 4.10.0"},

      # {:circuits_uart, "~> 1.4"},

      {:jason, "~> 1.2"},

      {:meeseeks, "~> 0.15.1"},

      {:decimal, "~> 2.0"},

      {:tesla, "~> 1.3"},

      {:castore, "~> 0.1.0"},

      # {:rustler, git: "git://github.com/rusterlium/rustler.git" branch: "master", sparse: "rustler_mix"},

      {:rustler, "~> 0.22.0-rc.1", override: true}

    ]

  end

I’d try both things:

  • Use the commented out line in your mix.exs:
{:rustler, git: "git://github.com/rusterlium/rustler.git" branch: "master", sparse: "rustler_mix"},
rustler = {git = "git://github.com/rusterlium/rustler.git", branch="master"}
5 Likes

(Gah, I’m sorry to double post, I messed up making an edit.)

You can override the version rustler is looking for with an ENV variable:

$ RUSTLER_NIF_VERSION=2.15 mix compile

This is the easiest way to make progress (I’ve found) without having to re-target any deps or anything.

Am I missing some downsides to doing this? I think rustler works just fine targeting the older nif version in my experience.

1 Like

The reason the approach above has been recommended is just one: rustler maintainers seem to be active again and we can expect a stable next release soon.

If not for that factor then I’d go with what you recommended.

Ah OK, that makes perfect sense and I didn’t realize. Thanks for letting me know :purple_heart: