Introducing Nx, Lambda days 2021

I know nothing about this subject but every time I listen to José I’m impressed by him.
José, você é o cara !

I had absolutely zero knowledge in machine learning, and that was actually a pretty good introductory class, despite its goal being only to show Nx features.

For those having some troubles compiling EXLA on Ubuntu 20.4, I found those commands to be helpful.

$ sudo apt install python3-pip
$ pip3 install numpy
$ cd /usr/bin
$ sudo ln -s python3 python

$ asdf plugin add bazel
$ asdf install bazel 3.1.0
$ asdf global bazel 3.1.0

The link is because it’s looking for python, not python3.

UPDATE: This is not the recommended way, see next post. It’s just a hack to get EXLA compilation working.

Bazel is also available as a plugin for asdf-vm. You need to use 3.1.0 instead of the latest 4.0.0

And now with all in place, I need to find what to do with it

For MacOs Mojave asdf wa the simplst solution for bazel too.

I have worked with Python for a few years and I can absolutely say this is not a good idea:

$ # DON'T DO THIS
$ cd /usr/bin
$ sudo ln -s python3 python # :(

You should instead make a virtual environment and associate it with whatever directory you occupy while playing with nx.

For example, here is how I do it using pyenv (GitHub - pyenv/pyenv: Simple Python version management · GitHub):

$ mkdir nx_fun
$ cd nx_fun
$ pyenv virtualenv 3.9-dev nx
$ pyenv local nx
$ pip install numpy

If you have set up pyenv properly then being in the nx_fun directory means python automatically points to the python3 shim for your virtual environment.

It is dangerous to override system python files which are often used under-the {:exla, “~> 0.1.0-dev”, github: “elixir-nx/nx”, sparse: “exla”},
{:nx, “~> 0.1.0-dev”, github: “elixir-nx/nx”, sparse: “nx”, override: true}-hood by a surprising number of packages. I advise never touching your system python files (let apt or yum or apk or brew handle those) and always use a virtual environment when you want to pip install.

The bazel instructions are good though. I also recommend asdf.

Update: I got a chance to try this and I want to add something. If using pyenv, you need to do pyenv global nx (if you followed the instructions above) in order for mix deps.compile exla to find numpy. I still recommend doing it this way since you can easily switch you global python version back when you are finished playing with nx. It’s still compiling for me…

I am not a python guy, I will follow your advice

Thanks for reporting.

It took 7 hours for EXLA to compile on my laptop…

Ouch.. that hurts. I had to go through at least 7 attempts before successfully compiling EXLA, each time with different errors. As last hope I kept insisting on mix compile and then after three rounds of it I could get it compiled.
I’m on Ubuntu 20.04, the successfull compilation attempt used bazel 3.2.0 through bazelisk (installed with npm) and I used python_is_python3 apt package to alias python.
An attempt to compile with cuda support destroyed a first successful build () and I had to try half the afternoon to get it compiled back again. The compilation process takes about 20 minutes on my machine.

Hi,

If you all can report to me some of your compilation problems and troubleshoot steps, I can put together a comprehensive guide for building EXLA so there are no more issues. Also please note that we fortunately no longer have the NumPy dependency

Is there a good article about the mechanisms used in the demo? Why do we use dot, why is the loss function implemented that way, why the grad is implemented that way, etc. And why do we add a single hidden layer, and why we add it, etc.

I can find a lot of tutorials that explains how to code it, just as José did, but not an article that explains the choices in a not-too-complicated way. Then there are the hard-math papers. I mean I am not a maths person ; I don’t mind diving into maths when it is necessary but all docs I found expect me to be able to reduce integrals and multiply matrices like I breathe.

I understand that maybe this topic assumes the developers have a decent math knowledge, but in the end, the code is able to predict numbers (i.e. it works) with just 2-3 math functions so that should not be that hard to grasp with a good explanation.