Tensorflex: Tensorflow bindings for Elixir

anshuman23 · June 14, 2018, 1:55pm

Hello all,

I have been working on my proposed project called Tensorflex as part of Google Summer of Code 2018.. Tensorflex can be used for making predictions from input data on pre-trained Tensorflow models. Training as of now is only supported in the Python API (also C++ at a low-level) but Tensorflex fully supports Inference and can be used for making predictions. Moreover, Tensorflex is being worked on and improved everyday so if there are any particular features anyone has in mind, please feel free to get in touch with me!

As an introduction to using Tensorflex for making predictions from saved models, I have written a blog post: http://www.anshumanc.ml/gsoc/2018/06/14/gsoc/

To follow development, make sure to watch the Github repository here: https://github.com/anshuman23/tensorflex

Cheers!

juhalehtonen · June 14, 2018, 6:24pm

Great! I was really excited to see this on the Google Summer of Code list of projects, glad to see it come to fruition You mentioned that training via Tensorflex is not supported yet. Do you have any timeline in mind for this feature to land?

anshuman23 · June 14, 2018, 9:13pm

Hello Juha! Thanks for the vote of confidence

So any bindings for Tensorflow can only be written around the C API, which is publicly exposed by Google. Unfortunately, they have not added Training functionality to the C API yet. So we will have to wait and see when they do that and then we can support it in Tensorflex too. Also, most other Tensorflow bindings, such as the ones officially written for Golang, only support Inference.

CptnKirk · June 14, 2018, 9:24pm

I’m interested in this project. But I’d be very interested in this as a case study of crafting Elixir bindings to C/C++ things. There’s plenty of FUD around NIFs and native binding in Erlang/Elixir. Yet this type of thing is ubiquitous in other languages.

Would love to know:

How you got started
What resources you used
Pitfalls
Performance
Why not wrap all the things?

anshuman23 · June 15, 2018, 5:25pm

Hello Jim! I completely agree and I’d love to help out.

To do justice to your question, I will post a detailed answer as a subsequent blog post soon and share the link here.

Cheers!

micahrye · June 17, 2018, 1:35am

With respect to C/C++ and NIFs I wonder if this would be a great project to also use Rust. There is the Rustler project that would be a good place to start. The FUD of NIFs is that they potentially blowup the ERTS, and hence fault tolerance.

Rust is able to handle runtime exceptions in a way that can handle such exceptions and provide safety, such that the Rust NIF does not take down the Beam.

CptnKirk · June 17, 2018, 4:34am

But most every library we’d be wrapping is C/C++. Here’s where the FUD kicks in. Rust is a new and another thing I’d have to learn. Also, from my limited knowledge of Rust, that while you can call C/C++ you give up safety by doing so.

So while you could write your own Rust lib for the fast parts and use Rustler to help with Elixir bindings, I’m not sure Rust helps in this case.

But I hear about Rust so it would be nice for someone to set the record straight.

OvermindDL1 · June 19, 2018, 6:46pm

That’s about right, Rust gives safety within rust, so it could make the interface safer, but it won’t do anything for the code running in another language, and if it is a simple wrapping anyway, not much point in using Rust then.

f34nk · June 20, 2018, 10:00am

Alternatively you could implement the binding as a C-Node.
There is a package called Nodex.

From Erlang’s point of view, the C node is treated like a normal Erlang node.

This way you still have fault tolerance.

froggrog · June 21, 2018, 1:09pm

You create a useful product, thank you! I recently found a similar project, it’s called extensor. Maybe you should join forces.

mat_garland_1 · July 16, 2018, 12:47am

I am really looking forward to this, thanks @anshuman23 for taking this on board.

It would be perfect if one were able to hand off data to an Elixir process -> Tensor flow / flex ML pipeline and then use that trained data. I am particularly interested in the text classification / LSTM / Inference areas of ML for bots. It sounds like this is exactly what you are working on?

anshuman23 · July 19, 2018, 6:18pm

Hi @mat_garland_1! Yes Tensorflex supports Inference for any graph created with Tensorflow. Since you mentioned LSTMs, I’m actually working on an example at the moment which will cover RNNs and LSTMs with Tensorflex. It should be up on the Github repository in a day or so. Will also try and write a blog post if it helps

CptnKirk · July 19, 2018, 10:21pm

http://erlang.org/doc/man/slave.html

You could also start a slave node. Execute your NIFs within the slave. With slaves you can also achieve fault tolerance. You’d also avoid the need for C code scaffolding. Assuming you got the NIF from a 3rd party you could avoid maintaining non-elixir code within your codebase.

mat_garland_1 · July 21, 2018, 8:40am

Thank you so much @anshuman23, I’ll certainly check it out! Great work!

anshuman23 · July 22, 2018, 11:06am

Hello!
@mat_garland_1 and others who might be interested, here is the link for the PR which showcases how to use an RNN-LSTM model for sentiment analysis in Tensorflex. I am going to be working on documentation and a blog post for this as well as the Inception model over the next week, but the description for this PR is sufficiently detailed to get a general idea in the meanwhile:

github.com/anshuman23/tensorflex

Added RNN (LSTM) example

anshuman23:master ← anshuman23:dev

opened 10:59AM - 22 Jul 18 UTC

anshuman23

+63 -0

This example is going to be incorporated into a more detailed blog post, but her…e is a brief idea: - The Recurrent Neural Network utilizes Long-Short-Term-Memory (LSTM) cells for holding the state for the data flowing in through the network - In this example, we utilize the LSTM network for sentiment analysis on movie reviews data in Tensorflex. The trained models are originally created as part of an online tutorial [(source)](https://www.oreilly.com/learning/perform-sentiment-analysis-with-lstms-using-tensorflow) and are present in a Github repository [here](https://github.com/adeshpande3/LSTM-Sentiment-Analysis). To do sentiment analysis in Tensorflex however, we first need to do some preprocessing and prepare the graph model (`.pb`) as done multiple times before in other examples. For that, in the `examples/rnn-lstm-example` directory there are two scripts: `freeze.py` and `create_input_data.py`. Prior to explaining the working of these scripts you first need to download the original saved models as well as the datasets: - For the model, download from [here](https://github.com/adeshpande3/LSTM-Sentiment-Analysis/raw/master/models.tar.gz) and then store all the 4 model files in the `examples/rnn-lstm-example/model` folder - For the dataset, download from [here](https://github.com/adeshpande3/LSTM-Sentiment-Analysis/raw/master/training_data.tar.gz). After decompressing, we do not need all the files, just the 2 numpy binaries `wordsList.npy` and `wordVectors.npy`. These will be used to encode our text data into `UTF-8` encoding for feeding our RNN as input. Now, for the Python two scripts: `freeze.py` and `create_input_data.py`: - `freeze.py`: This is used to create our `pb` model from the Python saved checkpoints. Here we will use the downloaded Python checkpoints' model to create the `.pb` graph. Just running `python freeze.py` after putting the model files in the correct directory will do the trick. In the same `./model/` folder, you will now see a file called `frozen_model_lstm.pb`. This is the file which we will load into Tensorflex. In case for some reason you want to skip this step and just get the loaded graph here is a Dropbox [link](https://www.dropbox.com/s/xp1bphy0k40v5r6/frozen_model_lstm.pb?dl=0) - `create_input_data.py`: Even if we can load our model into Tensorflex, we also need some data to do inference on. For that, we will write our own example sentences and convert them (read encode) to a numeral (`int32`) format that can be used by the network as input. For that, you can inspect the code in the script to get an understanding of what is happening. Basically, the neural network takes in an input of a `24x250` `int32` (matrix) tensor created from text which has been encoded as `UTF-8`. Again, running `python create_input_data.py` will give you two `csv` files (one indicating positive sentiment and the other a negative sentiment) which we will later load into Tensorflex. The two sentences converted are: - Negative sentiment sentence: _That movie was terrible._ - Positive sentiment sentence: _That movie was the best one I have ever seen._ Both of these get converted to two files `inputMatrixPositive.csv` and `inputMatrixNegative.csv` (by `create_input_data.py`) which we load into Tensorflex next. __Inference in Tensorflex:__ Now we do sentiment analysis in Tensorflex. A few things to note: - The input graph operation is named `Placeholder_1` - The output graph operation is named `add` and is the eventual result of a matrix multiplication. Of this obtained result we only need the first row - Here the input is going to be a integer valued matrix tensor of dimensions `24x250` representing our sentence/review - The output will have 2 columns, as there are 2 classes-- for positive and negative sentiment respectively. Since we will only be needing only the first row we will get our result in a `1x2` vector. If the value of the first column is higher than the second column, then the network indicates a positive sentiment otherwise a negative sentiment. All this can be observed in the original repository in a Jupyter notebook [here](https://github.com/adeshpande3/LSTM-Sentiment-Analysis): ```elixir iex(1)> {:ok, graph} = Tensorflex.read_graph "examples/rnn-lstm-example/model/frozen_model_lstm.pb" {:ok, %Tensorflex.Graph{ def: #Reference<0.713975820.1050542081.11558>, name: "examples/rnn-lstm-example/model/frozen_model_lstm.pb" }} iex(2)> Tensorflex.get_graph_ops graph ["Placeholder_1", "embedding_lookup/params_0", "embedding_lookup", "transpose/perm", "transpose", "rnn/Shape", "rnn/strided_slice/stack", "rnn/strided_slice/stack_1", "rnn/strided_slice/stack_2", "rnn/strided_slice", "rnn/stack/1", "rnn/stack", "rnn/zeros/Const", "rnn/zeros", "rnn/stack_1/1", "rnn/stack_1", "rnn/zeros_1/Const", "rnn/zeros_1", "rnn/Shape_1", "rnn/strided_slice_2/stack", "rnn/strided_slice_2/stack_1", "rnn/strided_slice_2/stack_2", "rnn/strided_slice_2", "rnn/time", "rnn/TensorArray", "rnn/TensorArray_1", "rnn/TensorArrayUnstack/Shape", "rnn/TensorArrayUnstack/strided_slice/stack", "rnn/TensorArrayUnstack/strided_slice/stack_1", "rnn/TensorArrayUnstack/strided_slice/stack_2", "rnn/TensorArrayUnstack/strided_slice", "rnn/TensorArrayUnstack/range/start", "rnn/TensorArrayUnstack/range/delta", "rnn/TensorArrayUnstack/range", "rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3", "rnn/while/Enter", "rnn/while/Enter_1", "rnn/while/Enter_2", "rnn/while/Enter_3", "rnn/while/Merge", "rnn/while/Merge_1", "rnn/while/Merge_2", "rnn/while/Merge_3", "rnn/while/Less/Enter", "rnn/while/Less", "rnn/while/LoopCond", "rnn/while/Switch", "rnn/while/Switch_1", "rnn/while/Switch_2", "rnn/while/Switch_3", ...] ``` First we will try for positive sentiment: ```elixir iex(3)> input_vals = Tensorflex.load_csv_as_matrix("examples/rnn-lstm-example/inputMatrixPositive.csv", header: :false) %Tensorflex.Matrix{ data: #Reference<0.713975820.1050542081.13138>, ncols: 250, nrows: 24 } iex(4)> input_dims = Tensorflex.create_matrix(1,2,[[24,250]]) %Tensorflex.Matrix{ data: #Reference<0.713975820.1050542081.13575>, ncols: 2, nrows: 1 } iex(5)> {:ok, input_tensor} = Tensorflex.int32_tensor(input_vals, input_dims) {:ok, %Tensorflex.Tensor{ datatype: :tf_int32, tensor: #Reference<0.713975820.1050542081.14434> }} iex(6)> output_dims = Tensorflex.create_matrix(1,2,[[24,2]]) %Tensorflex.Matrix{ data: #Reference<0.713975820.1050542081.14870>, ncols: 2, nrows: 1 } iex(7)> {:ok, output_tensor} = Tensorflex.float32_tensor_alloc(output_dims) {:ok, %Tensorflex.Tensor{ datatype: :tf_float, tensor: #Reference<0.713975820.1050542081.15363> }} ``` We only need the first row, the rest do not indicate anything: ```elixir iex(8)> [result_pos | _ ] = Tensorflex.run_session(graph, input_tensor,output_tensor, "Placeholder_1", "add") [ [4.483788013458252, -1.273943305015564], [-0.17151066660881042, -2.165886402130127], [0.9569928646087646, -1.131581425666809], [0.5669126510620117, -1.3842089176177979], [-1.4346938133239746, -4.0750861167907715], [0.4680981934070587, -1.3494354486465454], [1.068990707397461, -2.0195648670196533], [3.427264451980591, 0.48857203125953674], [0.6307879686355591, -2.069119691848755], [0.35061028599739075, -1.700657844543457], [3.7612719535827637, 2.421398878097534], [2.7635951042175293, -0.7214710116386414], [1.146680235862732, -0.8688814640045166], [0.8996094465255737, -1.0183486938476563], [0.23605018854141235, -1.893072247505188], [2.8790698051452637, -0.37355837225914], [-1.7325369119644165, -3.6470277309417725], [-1.687785029411316, -4.903762340545654], [3.6726789474487305, 0.14170047640800476], [0.982108473777771, -1.554244875907898], [2.248904228210449, 1.0617655515670776], [0.3663095533847809, -3.5266385078430176], [-1.009346604347229, -2.901120901107788], [3.0659966468811035, -1.7605335712432861] ] iex(9)> result_pos [4.483788013458252, -1.273943305015564] ``` Thus we can clearly see that the RNN predicts a positive sentiment. For a negative sentiment, next: ```elixir iex(10)> input_vals = Tensorflex.load_csv_as_matrix("examples/rnn-lstm-example/inputMatrixNegative.csv", header: :false) %Tensorflex.Matrix{ data: #Reference<0.713975820.1050542081.16780>, ncols: 250, nrows: 24 } iex(11)> {:ok, input_tensor} = Tensorflex.int32_tensor(input_vals,input_dims) {:ok, %Tensorflex.Tensor{ datatype: :tf_int32, tensor: #Reference<0.713975820.1050542081.16788> }} iex(12)> [result_neg|_] = Tensorflex.run_session(graph, input_tensor,output_tensor, "Placeholder_1", "add") [ [0.7635725736618042, 10.895986557006836], [2.205151319503784, -0.6267685294151306], [3.5995595455169678, -0.1240251287817955], [-1.6063352823257446, -3.586883068084717], [1.9608432054519653, -3.084211826324463], [3.772461414337158, -0.19421455264091492], [3.9185996055603027, 0.4442034661769867], [3.010765552520752, -1.4757057428359985], [3.23650860786438, -0.008513949811458588], [2.263028144836426, -0.7358709573745728], [0.206748828291893, -2.1945853233337402], [2.913491725921631, 0.8632720708847046], [0.15935257077217102, -2.9757845401763916], [-0.7757357358932495, -2.360766649246216], [3.7359719276428223, -0.7668198347091675], [2.2896337509155273, -0.45704856514930725], [-1.5497230291366577, -4.42919921875], [-2.8478822708129883, -5.541027545928955], [1.894787073135376, -0.8441318273544312], [0.15720489621162415, -2.699129819869995], [-0.18114641308784485, -2.988100051879883], [3.342879056930542, 2.1714375019073486], [2.906526565551758, 0.18969044089317322], [0.8568912744522095, -1.7559258937835693] ] iex(13)> result_neg [0.7635725736618042, 10.895986557006836] ``` Thus we can clearly see that in this case the RNN indicates negative sentiment! Our model works :D

IRLeif · September 29, 2018, 7:39pm

Thank you for creating this library, @anshuman23.

I recently changed my position at work from business developer to data scientist and I’m in the process of digging into a bunch of tools for data wrangling and machine learning.

Python is one of the most common programming languages for data science, due to extensive and robust libraries, such as pandas, NumPy, scikit-learn, TensorFlow, Keras and TensorForce. Those are the primary tools that we use at my workplace at the moment.

I’ll be looking into using Tensorflex as an alternative for deploying machine learning models.

One thing I’m curious about: When you created Tensorflex, did you envision for it to become to Elixir sort of what Keras is to Python? That is, not just for model-serving, but also for training artificial neural networks, etc.—A more generic abstraction layer on top of TensorFlow, CNTK and Theano.

Elixir with it’s highly concurrent capabilities, focus on data transformation (e.g. data pipelines) and strong support for meta-programming, lends itself very well to data science and machine learning. It makes me very excited to see that other people have similar thoughts.

There is an incredible amount of potential for Elixir within data science and machine learning.