I hope it’s not the 2nd of september 
1 Like
Airing the 2 of February
Also i thought “airing” meant “diffused through the air”, i.e. broadcasted, so “released”. Not “recorded”.
1 Like
Well, I am not an english native speaker, as my name suggest. I am speaking Voltaire’s lingua (Like You too?).
Like You too?
Sure 
1 Like
Just announced on Thinking Elixir podcast #034 José Valim reveals Project Nx - Thinking Elixir
Awesome! 
10 Likes
Listening now
2 Likes
Wrote down some notes. Please correct if something is wrong or missing, took these right while listening
Nx = Numerical Elixir
Everything is WIP, baby steps!
In a nutshell
Nx is a project that consists of a core library (Nx) and other pieces that depend on the core library. Together, they introduce things very much related to machine learning and associated fields. Nx introduces the ability to work with tensors, boosts Elixir’s ability and performance when working with numbers, and makes it possible to compile code for the GPU.
The work has been ongoing for 3 months. Everything is still in flux, but the following were mentioned.
There are 3 fronts where work is happening:
1) Nx library (core)
- Library for multidimensional arrays that are often called tensors.
- Tensors are typed, not in type-system way but unsigned integers or float32 or float64 etc.
- Earlier José made PR to support float16 in Erlang.
- If you come from Python, it can be thought of as kind of like Numpy. Long way to go but working on that.
- Written in pure Elixir
- Tensors represented as large binaries
- The “slowness in Elixir” with numbers is generally due to immutability and copying etc. This is addressed through a module that is part of Nx called Numerical Definitions
2) Numerical Definitions
- Module in Nx
- Looks like Elixir code
- Can define functions with
defn. That’s a numerical definition - In the definition you write with a subset of Elixir. Elixir kernel is replaced with Numerical kernel.
- In this, operators + and - can only work with numbers and tensors.
- In numdefs, we are not executing them immediately. We are building a computation graph of all tensor operations we want to do.
- Now when you want to calculate X, fuse all operations together to pass through the tensor only once, which reduces copying considerably
- Kind of like Elixir Streams but happens on syntax level inside
defn - Defn allows for “automatic differentiation”. The problem that this solves is essentially loss function minimization in things like when creating neural networks etc.
3) EXLA
- Based on Google XLA (accelerated linear algebra)
- Nx team wrote wrote Bindings to XLA, which is part of Tensorflow repo
- You can give it a computation graph and exla compiles it to run efficiently on CPU or GPU
- You can change the compiler. In other words, the compiler backend is pluggable, so other compilers aside from XLA can also be integrated with
- XLA compiler compiles numerical definitions to run on the GPU.
- What this means is subset of Elixir is running on the GPU
- Small subset of Elixir: still contains things like pipe operators and conditionals, but f.ex. limited pattern matching (only tuples)
More information will be available on February 17 at the Lambda Days 2021 conference where José will be speaking.
73 Likes
“Hey, what did you do over Christmas?”
me: “I drank too much and tried to read a book”
Jose Valim: “I rewrote my language’s kernel to afford tensorflow-like operations for efficient ML operations”
42 Likes
One really great part of this is that very few changes to Elixir itself were required. Nx is built on top what is already there in Elixir. It’s really good of example of the language extensibility we can have with meta programming and that you don’t have to be an Elixir core contributor to build libraries like this.
40 Likes
Listening now and i’am really excited, wow. 
2 Likes
Does that mean that it can use pattern matching with records given they’re translated to tuples at compile time?
1 Like
I like the GPU part sooooo much!
If done right (as usual from JV and the team) it would be “THE THING”!
Because there’s plenty to grow from here.
2 Likes
Ok that is good to hear. I imagined redefining the + operator would be quite a fundamental change.
1 Like
I wish it was named TNT (e.g. this new thing from Jose’s benchmark) and it would boom like TNT too 
3 Likes
Yup, here’s a quick example:
as you can see the implementation is quite straightforward!
8 Likes
A great summary was already posted above. Additionally adding a story.
After looking at the benchmarks, A person guessed they are doing it on GPU, sshhh keep it secret… then he was invited to the project!
2 Likes
I am SUPER excited about Nx 
I feel that Elixir is a great language in which to express numerical computation of the sort that is done in machine learning, just we didn’t have the right low-level support for it. That is, until Nx!
3 Likes
Does it work only on Nvidia GPUs or also on AMD ones?
1 Like
Afaik most tooling in machine learning does work with CUDA, so Nvidia GPUs.
4 Likes
XLA has a ROCm backend targeting AMD GPUs so the XLA Nx backend will support AMD. I’ve tested most of the project on an AMD GPU and it works, but we primarily test on Nvidia GPUs so there may be more issues when working with AMD GPUs. I also know there are only a few ROCm enabled AMD GPUs, but I’m not as familiar with the ecosystem.
6 Likes
