Elixir + [rust -> wasm] sample code?

[zeroexcuses]

@tessi

I am not sure how I can best help, but if you have any specific questions, feel free to ask

I don’t want to drag you into a flamewar or have you offend either party – but could you briefly summarize you choice of wasmer over wasmtime? I have played with both APIs slightly (as a Rust dependency), read a few github / reddit comments, understand that wasmtime seems very well integrated with Mozilla / cranelift team. But I really can’t figure out how they differ / when to use which.

For you, was this a flip of a coin, or were there technical differences that caused you to pick wasmer for wasmex ?

[autodidaddict]

@zeroexcuses

@autodidaddict

wasmCloud is designed for extensibility and polyglot, supporting actors in TinyGo, Rust, and AssemblyScript and it looks like lunatic’s SDK is specific to Rust

Would it be correct to say: stated another way, if you had full control over (1) choice of language and (2) choice of libraries your applications, then wasmCloud serves as a "docker replacement’ or sorts – instead of packaging an entire Linux image to throw on AWS EKS/ECS/Fargate, you can just build wasm binary (because you choose language + libraries that can target wasm), and you just provide a tiny wasm instead of a hunderds-of-MBs docker image ?

I prefer to think of wasm as the next evolution beyond docker, but you’re correct on all counts.

[autodidaddict]

@zeroexcuses

@tessi

I am not sure how I can best help, but if you have any specific questions, feel free to ask

I don’t want to drag you into a flamewar or have you offend either party – but could you briefly summarize you choice of wasmer over wasmtime? I have played with both APIs slightly (as a Rust dependency), read a few github / reddit comments, understand that wasmtime seems very well integrated with Mozilla / cranelift team. But I really can’t figure out how they differ / when to use which.

For you, was this a flip of a coin, or were there technical differences that caused you to pick wasmer for wasmex ?

I haven’t benchmarked the two of them, so I can’t say one is faster than the other. My experience with wasmer was that the API seemed slightly higher level, and a little bit easier to use than the wasmtime API. Other than that I didn’t see much difference.

[tessi]

@zeroexcuses

I don’t want to drag you into a flamewar or have you offend either party

I feel that our communities (rust, elixir) aren’t too much into flamewars It’s a super legitimate and good question.

It was really close between the two. I was mainly looking for the following traits:

• maintainability (stable API, backed by a stable team)
• speed of wasm execution
• community adoption and other libraries/code to learn from

maintainability / team

• wasmer and wasmtime both are pretty stable api-wise but still follow recent wasm developments
• both are backed by a company that does open source, wasmtime seemed to be closer to the rust core team – seen from a 5000km distance – due to both being connected to mozilla
• both projects seem to be in for the long run

speed of execution

There were many benchmarks flying around and it was a close call. At the day of decision wasmer was faster. But both were close enough of each other to make them practically indistinguishable for my use case.

community adoption

I was mainly looking for other programming language integrations. Wasmer had a ton of existing integrations to other host languages easy to find.

In the end, the existing language integrations of wasmer (in the hope to borrow some ideas) was the point that convinced me. To be honest, there was a timeframe some month ago were I was worried wasmer lost steam (compared to wasmtime) according to the number of commits/versions relreased. I was short before migrating to wasmtime.

Turned out that wasmer was secretly working on a rewrite which reduced the number of publicly visible commits. I wish that was communicated more clear and that their plan for their implementation was in the open. But things improved: they invited their community (we organize in a slack channel) to an early preview of their rewrite. It was relatively easy to upgrade to wasmer 1.0 (the rewrite) even before the official release. They listened a lot to feedback and were super helpful in answering questions or implementing fixes/helpers once one of us got stuck. We get frequent updates again and I’m happy.

Now, that I’m a little more connected to wasmer and know the devs behind it, their friendliness and support counts as another plus point for me

[zeroexcuses]

Thank you for your detailed reply. I’m new to wasmex, please bear with these trivial questions.

Can you verify if the following statements are correct:

wasmex allows wasm32 code to call arbitrary Elixir functions

imports = %{
  env: %{
    sum3: {:fn, [:i32, :i32, :i32], [:i32], fn (_context, a, b, c) -> a + b + c end},
  }
}
instance = start_supervised!({Wasmex, %{bytes: @import_test_bytes, imports: imports}})

{:ok, [6]} = Wasmex.call_function(instance, "use_the_imported_sum_fn", [1, 2, 3])

What is happening above is:

(1) we define a function in elixir (a + b + c)
(2) we can pass this fuhction to the wasm32 env
(3) when wasm32 invokes this function, the corresponding elixir code gets called

wasmex allows Elixir to read/write to wasm32 linear memory directly

{:ok, memory} = Wasmex.memory(instance, :uint8, 0)
index = 42
string = "hello, world"
Wasmex.Memory.write_binary(memory, index, string)

Here we are writing directly into the wasm32 memory, at index 42.

can we do dynamic code generation using wasmex ?

Ignoring Rust for a moment, and thinking in pure Elixir.

If I have an Elixir module that generates *.wast (the s-exp text format, not the binary format) code, can I load it in wasmex and execute it? The opens another route for doing dynamic code gen in Elixir, where for specialized routines, we can generate *.wast, copy the data into the linear memory, execute, and copy back. Is this possible?

[tessi]

@zeroexcuses

Thank you for your detailed reply. I’m new to wasmex, please bear with these trivial questions.

No worries, really. I am happy to help

@zeroexcuses

Can you verify if the following statements are correct:

wasmex allows wasm32 code to call arbitrary Elixir functions

imports = %{
  env: %{
    sum3: {:fn, [:i32, :i32, :i32], [:i32], fn (_context, a, b, c) -> a + b + c end},
  }
}
instance = start_supervised!({Wasmex, %{bytes: @import_test_bytes, imports: imports}})

{:ok, [6]} = Wasmex.call_function(instance, "use_the_imported_sum_fn", [1, 2, 3])

What is happening above is:

(1) we define a function in elixir (a + b + c)
(2) we can pass this fuhction to the wasm32 env
(3) when wasm32 invokes this function, the corresponding elixir code gets called

The short answer is: Yes, this is what happens.
The long answer is, that actually a little more happens (but you may not be interested in these details):

(1) we define a function in elixir (a + b + c)
(2) we can pass this function to the wasm32 env
(3) wasmex wraps the elixir function in a rust-wrapper function and passes it to wasm/wasmer
(4) we start a new wasmex GenServer (line 6 in your example code)
(5) the wasm part wants to call the sum3 function, our rust-wrapper function is called
(6) the rust-wrapper sends a message to the wasmex GenServer asking it to call the elixir function and waits for results
(7) the GenServer finds the elixir function in the imports-map, calls it in elixir land, and returns the result back to rust-land
(8) the wrapper function takes the function return values and converts them to wasm values and gives them back to WASM

If you want even more details, you can have a look at the PR implementing this feature Support imported functions by tessi · Pull Request #9 · tessi/wasmex · GitHub

@zeroexcuses

wasmex allows Elixir to read/write to wasm32 linear memory directly

{:ok, memory} = Wasmex.memory(instance, :uint8, 0)
index = 42
string = "hello, world"
Wasmex.Memory.write_binary(memory, index, string)

Here we are writing directly into the wasm32 memory, at index 42.

yes.
If you want you can directly manipulate bytes or pass arbitrary data (see this other person asking about passing data to wasm)

@zeroexcuses

can we do dynamic code generation using wasmex ?

Ignoring Rust for a moment, and thinking in pure Elixir.

If I have an Elixir module that generates *.wast (the s-exp text format, not the binary format) code, can I load it in wasmex and execute it? The opens another route for doing dynamic code gen in Elixir, where for specialized routines, we can generate *.wast, copy the data into the linear memory, execute, and copy back. Is this possible?

I have tested (and support) compiled wasm modules for wasmex only at the moment. But after looking at the wasmer docs, it seems they support WAT files. I found references that they use WAST files for internal testing, which gives me hope that WAST may just work.

So, it seems that wasmex may just be missing some tooling/config to support wat/wast. I’m very open for PRs or issues tracking this

[zeroexcuses]

@tessi

(line 6 in your example code)

Very minor nitpick: credit for the code goes to the wasmex documentation I copied/pasted from.

Thanks for your detailed & insightful response. I think I get the basics of wasmex now. Thanks again for your hard work in making this possible.

[echojoys]

Sorry to bother you, I’m currently learning the interaction between wasm and elixir. I am writing wasm in rust language, but it can only return the starting position of the address of the string, which is *const u8. I can’t get the length. What is the solution? Thanks again.

[tessi]

Hey @echojoys - apologies for my late answer. But I have good news! Back when you asked this question the Rust/WASM ecosystem wasn’t able to support multi-value returns yet. And we’d need multi-value returns to return the string address and length at the same tine.

This changed, however. Have a look at Return string form rust in Wasm - #11 by tessi where I tell you in another thread about the good news that Wasmex will ship with multi-value returns soon.