Because I needed to figure it out for my own sake, I also added an example of using Serde and Poison to the phoenix_rustler_integration github example.
This allows you to build up a data structure in Elixir, encode it as JSON and send it to the Rust function, do things with the equivalent structure in Rust with the Serde Macros, and then pass it back into Elixir, which can then decode it with Poison.
This is a LOT easier to do than the alternatives. Probably not quite as fast, but it depends on what your purposes are.
It could probably be changed to BSON or CBOR or what have you.
I'm thinking about it, and I'm not sure if what I just did is crazy or not. But it certainly was a lot simpler than having to manually decode the tuples or maps for any data structures with significant complexity. If you are doing heavy computation on the Rust side (which is why I am using it), and outputting JSON to be handled by the client, then maybe that is the main benefit. You can go DIRECTLY into the rust structure, and DIRECTLY into a suitable data structure for the client.