Tensorflex: Tensorflow bindings for Elixir

Great! I was really excited to see this on the Google Summer of Code list of projects, glad to see it come to fruition You mentioned that training via Tensorflex is not supported yet. Do you have any timeline in mind for this feature to land?

Hello Juha! Thanks for the vote of confidence

So any bindings for Tensorflow can only be written around the C API, which is publicly exposed by Google. Unfortunately, they have not added Training functionality to the C API yet. So we will have to wait and see when they do that and then we can support it in Tensorflex too. Also, most other Tensorflow bindings, such as the ones officially written for Golang, only support Inference.

I’m interested in this project. But I’d be very interested in this as a case study of crafting Elixir bindings to C/C++ things. There’s plenty of FUD around NIFs and native binding in Erlang/Elixir. Yet this type of thing is ubiquitous in other languages.

Would love to know:

• How you got started
• What resources you used
• Pitfalls
• Performance
• Why not wrap all the things?

Hello Jim! I completely agree and I’d love to help out.

To do justice to your question, I will post a detailed answer as a subsequent blog post soon and share the link here.

Cheers!

With respect to C/C++ and NIFs I wonder if this would be a great project to also use Rust. There is the Rustler project that would be a good place to start. The FUD of NIFs is that they potentially blowup the ERTS, and hence fault tolerance.

Rust is able to handle runtime exceptions in a way that can handle such exceptions and provide safety, such that the Rust NIF does not take down the Beam.

But most every library we’d be wrapping is C/C++. Here’s where the FUD kicks in. Rust is a new and another thing I’d have to learn. Also, from my limited knowledge of Rust, that while you can call C/C++ you give up safety by doing so.

So while you could write your own Rust lib for the fast parts and use Rustler to help with Elixir bindings, I’m not sure Rust helps in this case.

But I hear about Rust so it would be nice for someone to set the record straight.

That’s about right, Rust gives safety within rust, so it could make the interface safer, but it won’t do anything for the code running in another language, and if it is a simple wrapping anyway, not much point in using Rust then.

Alternatively you could implement the binding as a C-Node.
There is a package called Nodex.

From Erlang’s point of view, the C node is treated like a normal Erlang node.

This way you still have fault tolerance.

You create a useful product, thank you! I recently found a similar project, it’s called extensor. Maybe you should join forces.

I am really looking forward to this, thanks @anshuman23 for taking this on board.

It would be perfect if one were able to hand off data to an Elixir process → Tensor flow / flex ML pipeline and then use that trained data. I am particularly interested in the text classification / LSTM / Inference areas of ML for bots. It sounds like this is exactly what you are working on?