After doing a port from a c++ library to my project in phoenix I’ve seen that I need a faster way to run this algorithm and I found this amazing library:
Like C for Python, is Rust for Elixir (?)
I’m just begginer on Rust, but in all research it’s like C on performance (I’ve seen operative systems written on Rust), and Rust looks like Elixir in some way to write code (the matches for example)
What do you think to use it on big project?
Think that’s will be the beginnig to optimize actual libraries to do it a lot more faster?
I know that AppSignal has created a Rust integration, with the idea that the logging code is as fast as it needs to be, and that can be re-used for Ruby and Elixir (and in the future potentially other languages as well). I believe they also use rustler to integrate with it.
I heard @bruce is working on a GraphQL parser in Rust that’ll play nicely with Absinthe. Not sure what his approach will be but I’m sure it’ll be a good one.
At the risk of triggering members of the Rust Evangelism Strike Force I would say Use Nim, though I would have to admit that Rust appears to be better suited to handling the threading and low level interprocess communcating issues that Elixir and Erlang are used for.
I just left a single Use Nim comment in a Rust related post on HN and it got downvoted into oblivion, but other Nim advocates though came to post in a more agreeable manner.
The whole thread though is well worth reading as regards Rust.
I made an example to show the basics of putting Rust into a Phoenix project. Please let me know if you’re interested in seeing an example of another basic feature. I’m new to the languages and I’ve been writing these up as a way to share and mark down how I actually got things working.
Right now I just have the example provided by the originator of Rustler, but modified to work with Phoenix, and a couple of additional examples with using Elixir tuples and maps with Rust access.
@jc00ke That’s what I’m looking for actually! I hope he can achieve it! (Already exist a library for GraphQL on rust called juniper)
@vonH Oh thanks to share the Nim lang, looks pretty interesting, I also have “My Struggles with Rust”, the assing of variables and mut/ref variables handling and the usize on almost everything is making me crazy hahaha. Nim looks really good
@GregFreeman oh so many thanks, I think thats the missing example that people need, I think can be cool an example on a phoenix umbrella project
@OvermindDL1 You’re my hero, thats what I’m afraid, I was really sure that this interop need some rules to be implemented, with this now I can take more care on this “rules” and write better code
Because I needed to figure it out for my own sake, I also added an example of using Serde and Poison to the phoenix_rustler_integration github example.
This allows you to build up a data structure in Elixir, encode it as JSON and send it to the Rust function, do things with the equivalent structure in Rust with the Serde Macros, and then pass it back into Elixir, which can then decode it with Poison.
This is a LOT easier to do than the alternatives. Probably not quite as fast, but it depends on what your purposes are.
It could probably be changed to BSON or CBOR or what have you.
I’m thinking about it, and I’m not sure if what I just did is crazy or not. But it certainly was a lot simpler than having to manually decode the tuples or maps for any data structures with significant complexity. If you are doing heavy computation on the Rust side (which is why I am using it), and outputting JSON to be handled by the client, then maybe that is the main benefit. You can go DIRECTLY into the rust structure, and DIRECTLY into a suitable data structure for the client.
It’s simply amazing your aproach! I was thinking today how to pass a list of data to rust and you solve it for me hahahaha
Yeah maybe the main problem can be performance, but I think the big fault of performance its Poison, I have used it for a lot of json work and still very slow compared with c# (I know its not a correct comparison but after many time with Json.Net your mind keep this performance)
I think your repo its a good star for any that wants join phoenix and rust so many thanks
NIFs have other problems too. If you have an expensive NIF in your hot path even dirty schedulers wont help you.
We did lots of rsa_private_decrypt (30% of our requests) and the BEAM did not behave well at all under load. That is to say it behaved a lot like other languages
Latency was fluctuating, requests were being starved and it was only using 60% server CPU even thought the scheduler usage was 100%. The reason being the schedulers being held up waiting for NIFs who could not yield.
I always thought NIF was the answer to performance problem, however after converting this part of the code to using ports instead our application all of a sudden behaved as it was supposed to, mainly because the erlang schedulers were able to do what they are supposed to.
This made me more cautious of using NIFs for anything remotely long-running (=> 1ms) and rather using ports for those tasks. (and always measure, measure and measure)
What about using the Erlang External Term format instead of JSON? Integrations for Rust exists, and it would probably be faster to encode/decode than JSON, with less information being lost underway.
Interesting points. There is also https://github.com/sile/eetf
Do either of you know of a clear tutorial of how to set this up on the elixir/mix side to automate the compilation and generation of the rust libaries?
That’s one thing I seem to think are missing in both. There was enough information in Rustler for me to extend what needed to do with Phoenix. I got ports to work last night with some experimentation, but it was a lot of manual calls to rustc and setting flags in cargo.toml. Not automatic like the Rustler integration.
I’m happy to write a tutorial myself, I just need some guidance. I’ve only been using rust for about 3 weeks now, and Elixir only a little bit off and on until very recently.
Uh, that is definitely very costly and should not be done, not json, not bson, not cbor or anything of the sort. Rustler is a nif, it is designed for speed, thus you should use the NIF protocol, which is the ETF (Erlang Term Format), especially since Rustler already includes all necessary functions for accessing and modifying and working with it. JSON is not a fast protocol, why do you think Poison is often the bottleneck in web processing of super-fast requests.
Really though, stick with ETF. If you are manually decoding large ETF structures in Rust then you really need to build more helpers for it, or write a macro to auto-generate them.
members of the Rust Evangelism Strike Force I would say 
