Is there a supported way to compile + cache a rust dependency?

Questions / Help
, ,
1

Hi folks,

On of my team’s projects has a rust dependency, via a library that uses rustler to wrap a rust function. It hasn’t changed in a while, and it’s a pain to have an entire rust runtime dependency for this single lib.

We’re trying to find a way to remove the runtime from our deploy pipeline, and I’m wondering if there is a way to locally compile the dependency ahead of time and just include some kind of version-fixed binary.

Grateful for any suggestions, thanks.

3 Likes
2

We are working on a package for precompiling Rustler. It should be available within the next two weeks or so. :slight_smile:

28 Likes
3

Awesome news!!! :heart::blue_heart::purple_heart::yellow_heart::green_heart:

1 Like
4

Ooo this sounds interesting! Where is this being worked on?

1 Like
5

There is a proof of concept in the html5ever project that is being ported to a stand-alone library.

3 Likes
6

What I would find really fascinating is embedding a wasm engine, like wasmer or so, and expose the BEAM API to it (perhaps with a bit of massaging), and then you could easily compile native apps to WASM and load them in that instead. Something like wasmer could run it either fully interpreted, cranelift JIT compiled, or even LLVM native compiled for near native speed, without needing to compile it for each individual platform and beam bindings needed, especially if a more version independent API were exposed to it… This is an interesting idea… You’d probably want to precompile the wasm engine library for NIF access for each combination of beam VM and arch but that’s only one thing you’d need to do instead of every single possible library… Hmm…

Standalone process communicated via Ports or something similar is doable as well and honestly probably fine for many libraries, but most things made for NIF’s are because they want that speed, and WASM could probably get that close enough (at the very least for dev)…

3 Likes
7

If anybody else is curious, here is the package: GitHub - rusterlium/html5ever_elixir: NIF wrapper of html5ever using Rustler

3 Likes
8

Not seeing a PR or a branch t hat contains this new project Jose mentioned, I presume it’s being developed somewhere else.

9

If I understood correctly, Jose is porting this module to a library : https://github.com/rusterlium/html5ever_elixir/blob/master/lib/html5ever/precompiled.ex

3 Likes
10

Well, that’s an awesome response that I wasn’t expecting. Thanks for the heads up @josevalim :slight_smile:

I’ll be around in the first people to help test it if needed.

11

Much appreciated, thanks for the link.

12

Some files concerned:

1 Like
13

https://github.com/jayjun/rambo does also use precompiled binaries by default iirc.

2 Likes
14

Nice, thanks

15

Here is the library, very alpha version, likely some rough edges and changes incoming, but if you can’t wait you can already give it a try: GitHub - philss/rustler_precompiled: Use precompiled NIFs from trusted sources in your Elixir code

8 Likes
16

Thanks Jose, we’re giving this a try in our team this week, we’ll report back if as we learn more.