Dependencies for nx in Livebook on Windows

jshprentz

I tried the Launch Week Day 2 example on a newly installed Livebook v0.9.1 running on Windows 11. As in the Day 2 video, I created a new notebook and added a neural network smart cell. I confirmed the popup to add the kino_bumblebee and torchx dependencies and restart. After some time and much output, it fails.

The notebook setup created automatically is:

Mix.install(
  [
    {:kino_bumblebee, "~> 0.2.1"},
    {:torchx, "~> 0.5.1"}
  ],
  config: [nx: [default_backend: Torchx.Backend]]
)

The error message is:

==> torchx
could not compile dependency :torchx, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile torchx", update it with "mix deps.update torchx" or clean it with "mix deps.clean torchx"
** (Mix.Error) "nmake" not found in the path. If you have set the MAKE environment variable,
please make sure it is correct.

    (mix 1.14.2) lib/mix.ex:513: Mix.raise/2
    (elixir_make 0.7.6) lib/elixir_make/compiler.ex:120: ElixirMake.Compiler.cmd/5
    (elixir_make 0.7.6) lib/elixir_make/compiler.ex:97: ElixirMake.Compiler.make/2
    (elixir_make 0.7.6) lib/elixir_make/compiler.ex:53: ElixirMake.Compiler.compile/1
    (mix 1.14.2) lib/mix/task.ex:421: anonymous fn/3 in Mix.Task.run_task/4
    (mix 1.14.2) lib/mix/tasks/compile.all.ex:92: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.14.2) lib/mix/tasks/compile.all.ex:72: Mix.Tasks.Compile.All.compile/4
    #cell:setup:1: (file)

I notice that the video shows different dependencies: kino_bumblebee and exla.

What other dependencies and setup steps do I need to run the neural network examples in Livebook on Windows?

3 comments

/livebook #troubleshooting

0 1876 3

2023-04-16 07:39:13 UTC

Showing Posts 1 to 3

Show Best Posts
Show All (oldest first)
Show All (newest first)

josevalim

Creator of Elixir

You need to install Visual c++ Express on Windows: Visual Studio: IDE and Code Editor for Software Development

The error message should have said this. I investigate why it didn’t.

But generally speaking the error message, the ML tooling in general, even Python, focused on Linux, so consider using the Docker image too.

Post #1

jshprentz

Thanks. I will try the Docker image.

I was able to get many of the neural network features working after some trial and error. These steps led to success:

Install Microsoft C++ development tools.
Install CUDA 11.6
Install cuDNN 8.9.0
Disable Python aliases in Windows Settings/Manage App Execution Aliases
Apply Microsoft development environment variables to the Livebook notebook setup cell.
Add additional environment variables reported in error messages.
Click “Reconnect and setup” above the setup cell.

To capture the Microsoft development environment variables, run

cmd /K "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64

and type set to show all environment variables. Edit them into an Elixir map passed to Mix.install as system_env.

In response to error messages, I added CUDNN_ROOT, LIBTORCH_TARGET, and PYTHON.

With those changes, the setup cell begins with the following contents. Paths, versions, and unrelated environment variables may vary on other computers.

Mix.install(
  [
    {:kino_bumblebee, "~> 0.2.1"},
    {:torchx, "~> 0.5.1"}
  ],
  config: [nx: [default_backend: Torchx.Backend]],
  system_env: %{
    "LIBTORCH_TARGET" => "cu116",
    "CUDNN_ROOT" => "C:\\Program Files\\NVIDIA\\CUDNN\\v8.9",
    "PYTHON" => "C:\\Users\\Joel\\AppData\\Local\\Programs\\Python\\Python39\\python.exe",
    
    "ALLUSERSPROFILE" => "C:\\ProgramData",
    "APPDATA" => "C:\\Users\\Joel\\AppData\\Roaming",
    "CARBON_MEM_DISABLE" => "1",
    "ChocolateyInstall" => "C:\\ProgramData\\chocolatey",
    "ChocolateyLastPathUpdate" => "133196555883524332",
    "CommandPromptType" => "Native",
    "CommonProgramFiles" => "C:\\Program Files\\Common Files",
    "CommonProgramFiles(x86)" => "C:\\Program Files (x86)\\Common Files",
    "CommonProgramW6432" => "C:\\Program Files\\Common Files",
    "COMPUTERNAME" => "JOELS-DELL-XPS-",
    "ComSpec" => "C:\\Windows\\system32\\cmd.exe",
    "CUDA_PATH" => "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6",
    "CUDA_PATH_V11_6" => "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6",

I have not attempted to repeat this setup from scratch on a clean machine.

Post #2

josevalim

Creator of Elixir

Nice! If you are using the desktop app, there is a file that is executed once it boots to set env vars. You can find it here: GitHub - livebook-dev/livebook: Automate code & data workflows with interactive Elixir notebooks · GitHub - maybe you can set it once and then use notebooks as usual?

Post #3

— All posts loaded —