Project Nx - predictions and potential use cases

Quantum supremacy!

Which should be done outside of the application anyway.

In general - AI (NLP, NN, etc.) and engineering computations - stuff that the NumPy/SciPy and Julia are commonly used. However I worry that it will be really hard to win over scientist hearts. Julia has troubles with winning hearts over Python while being designed from day 1 to be used in such cases. I do not think that Elixir will succeed in that, but prove me wrong.

Outside of the typical AI and ML space, here some examples of Jax (one of Nx’s inspirations) thriving in a broad number of spaces:

• Molecular Dynamics Simulations
• Differentiable Cosmology
• Differentiable Path Tracing
• Neuroevolution

It seems as well that libraries like Jax are growing increasingly more popular for research. See DeepMind’s Jax Ecosystem

Of course, we are in our early stages, but hopefully we get to a point where people are comfortable adopting Elixir for some of their numerical work rather than shelling out to Python, Julia, R, MatLab, etc.

There are a lot of possibilities, it is only just the beginning!

Given the relative popularity of bitcoin traders in the Elixir community, no doubt some will be out chasing alpha for fun with Nx.

I know I’ve had some mild interest in using BEAM for games, if the GC latency and jitter is low enough, and float performance could be dramatically improved. Between Nx and BeamAsm, this might even become realistic for less-intensive games.

Personally, as a non-data scientist, I’m really excited about NumPy/Scikit behavior provided in the Elixir ecosystem. It means on my next personal project/side project (think startups) I won’t have to add another server to my infrastructure or another build stage to my dockerfile, etc… This extends my “simple” infrastructure timeline much farther into the future giving me the opportunity to focus on business goals. Maybe if mono-repos can now do both Web and Data Science the odds for success are improved.

I have been debating for a while about what the best why to include data science into an app is. Server via http, one dockerfile with multistage builds, hand-off to sagemaker, forget elixir and just build with flask or django… now i actually get to rethink this solely in the elixir context. Huge win. So yeah, hoping for NumPy/SciKit type behavior.

Apps grow and change. If the app grows enough to warrant hiring Data people and extracting the DataSci part to Python or Julia etc… it’s still a success that Elixir helped attain.

It took Elixir going into Machine Learning for my first post here! Kudos to Jose, Sean and everybody else who helped launch this in record time.

The fact that Nx will be like Numpy and have some feature parity will be awesome. Just FYI, Numpy has been around 2006 and it wasnt till 2011 (5 years later) that Numpy could spread its wings and be a productive tool (at least for me) speaks volumes on what Jose and team accomplished in three months!.

On the Thinking Elixir podcast, Jose kept talking about tensors. For math geeks, tensors are really generalized matroids. Matroids are a combinatorial structure that lives at the intersection of Optimization, Linear Algebra, and Graph Theory (Source: Wikipedia). Matroids have many applications in geometry, topology, combinatorial optimization, network theory, coding theory, and machine learning. That’s where I would look for use cases and applications.

Given what Jose was discussing around Softmax classifiers, one could deduce, he is talking about Deep Learning based computer vision use cases like image classification and object detection (e.g. boundary boxes). Computer vision has one of the richest set of use cases in AI.

• Protein structure prediction for medicine and vaccines
• Human Pose Estimation
• Image transformation like Snapchat filters
• Converting 2D images into 3D models.
• Medical Diagnostic Imaging analysis.

I have many more questions around the hardware side:

• How will Nx interface with CPU, GPU and TPU (Tensor Processing units)? I got excited when Jose talked about how there are flags for GPU based operations.

• Will Nx have support for GPU Acceleration using CUDA (proprietary) and OpenCL (open source)?

• What other tensor compilers are supported in the initial release of Nx?

• Today, Keras (higher level syntactic sugar wrapper for TensorFlow) is used mostly as TensorFlow is hard, so how does Nx play with Keras or TensorFlow? or even the next incumbent Pytorch?

• The next question would be around are are we talking about small datasets or large ones?

Also, Interested in seeing a comparison akin to a Python vs Elixir showdown (all things equal) for the same training and inference workloads on the same hardware.

Sorry for the long post. I am so hyped and excited for Nx, it feels like the dam burst for me.

I live in Toronto (the spiritual home of Deep Learning) and the 2019 NBA Champions aka Toronto Raptors.

You have my vote to fork a new topic or category page for Nx in this forum.

How will Nx interface with CPU, GPU and TPU (Tensor Processing units)? I got excited when Jose talked about how there are flags for GPU based operations.

As José mentioned in the podcast Nx can interface with CPU and GPUs. It’s all backend dependent. As an example the XLA backend handles all of the device stuff. It’ll be easier to make sense of when everything gets released. On that note, XLA is most optimized for compiling to TPUs, so it’s certainly something that could be supported in the future; however, we haven’t gotten to that point yet. TPUs are also a bit harder to test on considering they’re only available in GCP. If anybody has the time or resources and wants to help contribute on that front that’s also welcome

Will Nx have support for GPU Acceleration using CUDA (proprietary) and OpenCL (open source)?

Backends are flexible, so no reason it can’t have support for both.

What other tensor compilers are supported in the initial release of Nx?

XLA is the only one right now, but anything that can be surfaced to Elixir can probably be supported.

Today, Keras (higher level syntactic sugar wrapper for TensorFlow) is used mostly as TensorFlow is hard, so how does Nx play with Keras or TensorFlow? or even the next incumbent Pytorch?

Nx can’t interface directly with TensorFlow or PyTorch, although something like that would be possible with the addition of DLPack support.

The next question would be around are are we talking about small datasets or large ones?

Jax (thanks to XLA) holds the record for training ResNet-50 on ImageNet (~14 million images) in 29 seconds. I’d say this is probably more hardware/backend dependent than specifically Nx dependent.

It’s nice to see people so excited, I hope that answered your questions! Also recommend checking out the LambdaDays talk for more

Being able to write a data processing pipeline using GenStage/Flow/Broadway or similar and then not needing to dump the data somewhere for Python or R to process it I could see being useful.

Very helpful response Sean. It answers a lot of my questions. Thank you!

Subsequent thought:

if Nx has significant feature parity with Python’s Numpy and Jax (which is Numpy on Steroids) on its first few releases, it would be a massive game changer, for existing and new users of Elixir who today have to still use Python’s data analysis tools like Numpy etc. Basically, in the future, there is a possibility Elixir could do it all

One final question:

I got excited when Jose talked about the “graph” (which is typically used for computations on tensor data structures in deep learning frameworks). Just curious if ‘eager execution’ feature will be supported on Nx?

I know in PyTorch you define the graph at runtime, which allows you to go back and forth between planning and execution very easily. The ability to evaluate operations immediately, without compiling graphs explicitly, is eager execution.

The reason I mention this is because I find Eager execution allows you to prototype faster and create new types of architectures but at the cost of speed. This is akin to the difference between compiled and interpreted languages. It opens to the door to other types of use cases.

This used to be a big deal a few years ago, since TensorFlow used static graphs back then, requiring you to define the entire graph first before pushing data through. However, both Pytorch and TensorFlow frameworks now support eager execution by default, and this has since been adopted as the go-to industry standard. Just wondering if Nx will follow along.

… I know I should be more patient till Lambda days and Feb 17 could not come sooner!

I am looking to use some pre-trained models already built and test it with Nx and share my results here with the community.

No way. Miners are built with as low level as possible abstractions to squeeze out every ounce of performance. Yeah some are written in Go etc but they’re talking to C-based low level OpenCL drivers or even sometimes inline GCN assembler on AMD. Elixir + XLA will be very fast but is likely to show 5-10% perf gone for abstraction friendliness and that’s much too much for miners, because their power/capital costs are big and their revenues just a big bigger. In that scenario 5% faster can mean double profit. Remember miners have no use for what BEAM brings along other than possibly supervisors. They’re generally just one process doing one thing mega and simple as fast as possilble. Mining is not an architecturally complex problem. It’s a code-craft complex problem. Low level programming.