Compiling Torchx Nx backend on Windows

Elixir Framework Forums Nx Forum
,
1

Hi all,

I’m trying to get a new project off the ground on WIndows (client preference) that uses Nx (and Explorer, and Axon, and Scholar).

As far as I’ve understood, the only Nx.Backend that supports Windows currently is Torchx. Exla is only supported thourgh WSL, which is not a dependency I can expect on the final machines running this application.

After conquering some hurdles first (setting up cmake and making nmake available through running that pesky vcvarsall.bat script), I’m in a place where mix install doesn’t exit with a failure. But nonetheless I can’t use the torchx backend yet.

Looking at the output of mix install I assume the installation/build is still not successful:

00:35:20.769 [warning] The on_load function for module dll_loader_helper_beam_nif returned:
{:error,
 {:load_failed,
  ~c"Failed to load NIF library c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/dll_loader_helper_beam/priv/dll_loader_helper_beam: 'Unspecified error'"}}


00:35:20.778 [warning] The on_load function for module Elixir.Torchx.NIF returned:
{:undef,
 [
   {:dll_loader_helper_beam_nif, :add_dll_directory,
    ["c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/priv/libtorch"],
    []},
   {Torchx.NIF, :__on_load__, 0, [file: ~c"lib/torchx/nif.ex", line: 7]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: ~c"code_server.erl", ...]}
 ]}


00:35:20.769 [error] Process #PID<0.3238.0> on node :"24gbudsi-livebook_trekzrcs@WinDev2401Eval" raised an exception
** (UndefinedFunctionError) function :dll_loader_helper_beam_nif.add_dll_directory/1 is undefined (module :dll_loader_helper_beam_nif is not available)
    (dll_loader_helper_beam 1.2.0) :dll_loader_helper_beam_nif.add_dll_directory("c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/priv/libtorch")
    lib/torchx/nif.ex:7: Torchx.NIF.__on_load__/0
    (kernel 9.2.1) code_server.erl:1398: anonymous fn/1 in :code_server.handle_on_load/5

Full output:

Summary 
Microsoft (R) Program Maintenance Utility Version 14.38.33134.0
Copyright (C) Microsoft Corporation.  All rights reserved.

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- The CXX compiler identification is MSVC 19.38.33134.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx86/x86/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using PRIV_DIR: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/dll_loader_helper_beam/priv
-- Using ERTS_INCLUDE_DIR: c:/Users/User/scoop/apps/erlang/26.2.2/erts-14.2.2/include/
-- Configuring done (7.3s)
-- Generating done (0.0s)
-- Build files have been written to: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/dll_loader_helper_beam/build
[ 50%] Building CXX object CMakeFiles/dll_loader_helper_beam.dir/c_src/dll_loader_helper_beam.cpp.obj
dll_loader_helper_beam.cpp
[100%] Linking CXX shared library dll_loader_helper_beam.dll
[100%] Built target dll_loader_helper_beam
-- Installing: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/dll_loader_helper_beam/priv/dll_loader_helper_beam.lib
-- Installing: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/dll_loader_helper_beam/priv/dll_loader_helper_beam.dll
===> Analyzing applications...
===> Compiling dll_loader_helper_beam
==> dll_loader_helper
Compiling 1 file (.ex)
Generated dll_loader_helper app
==> torchx

Microsoft (R) Program Maintenance Utility Version 14.38.33134.0
Copyright (C) Microsoft Corporation.  All rights reserved.

11 File(s) copied
-- Building for: Visual Studio 17 2022
-- The C compiler identification is MSVC 19.38.33134.0
-- The CXX compiler identification is MSVC 19.38.33134.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Torch: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/torchx/cache/libtorch-2.0.0-cpu/lib/torch.lib  
-- Configuring done (17.9s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    LIBTORCH_BASE


-- Build files have been written to: C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/cmake
MSBuild version 17.8.3+195e7f5a3 for .NET Framework

  Checking File Globs
  1>Checking Build System
  Building Custom Rule C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/torchx/CMakeLists.txt
  torchx.cpp
     Creating library C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/cmake/Release/torchx.lib and object C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/cmake/Release/torchx.exp
  torchx.vcxproj -> C:\Users\User\AppData\Local\mix\Cache\installs\elixir-1.16.1-erts-14.2.2\a66ba6f8744462a3fd90d088b0a6c530\_build\dev\lib\torchx\cmake\Release\torchx.dll
  Building Custom Rule C:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/deps/torchx/CMakeLists.txt
1 File(s) copied
Compiling 3 files (.ex)

00:35:20.769 [warning] The on_load function for module dll_loader_helper_beam_nif returned:
{:error,
 {:load_failed,
  ~c"Failed to load NIF library c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/dll_loader_helper_beam/priv/dll_loader_helper_beam: 'Unspecified error'"}}


00:35:20.778 [warning] The on_load function for module Elixir.Torchx.NIF returned:
{:undef,
 [
   {:dll_loader_helper_beam_nif, :add_dll_directory,
    ["c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/priv/libtorch"],
    []},
   {Torchx.NIF, :__on_load__, 0, [file: ~c"lib/torchx/nif.ex", line: 7]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: ~c"code_server.erl", ...]}
 ]}


00:35:20.769 [error] Process #PID<0.3238.0> on node :"24gbudsi-livebook_trekzrcs@WinDev2401Eval" raised an exception
** (UndefinedFunctionError) function :dll_loader_helper_beam_nif.add_dll_directory/1 is undefined (module :dll_loader_helper_beam_nif is not available)
    (dll_loader_helper_beam 1.2.0) :dll_loader_helper_beam_nif.add_dll_directory("c:/Users/User/AppData/Local/mix/Cache/installs/elixir-1.16.1-erts-14.2.2/a66ba6f8744462a3fd90d088b0a6c530/_build/dev/lib/torchx/priv/libtorch")
    lib/torchx/nif.ex:7: Torchx.NIF.__on_load__/0
    (kernel 9.2.1) code_server.erl:1398: anonymous fn/1 in :code_server.handle_on_load/5
Generated torchx app

I should add that I’m bulding with Visual Studio 2022, and not the 2019 edition, as stated in the installation instructions (both “Microsoft Build Tools 2019” and “Microsoft Visual C++ 2019 Redistributable” are mentioned). I’m actually using the free WIndows 11 development environment images in VirtualBox, which comes with the Visual Studio 2022 installation.

Does this error seems familiar to anyone? Anything I’ve forgotten. Or maybe switch to Visual Studio 2019, instead of 2022?

2

Can you try Torch v0.7? Although I recommend double checking the option for WSL. The windows side of tooling for ML is often lacking, even in Python (for example Jax doesn’t support it either and only recently gained experimental status). Another option is Ortex. You won’t be able to use Bumblebee models (at the moment) but you can export Python ones.

3

That indeed is a step forward! I was on a slightly older version of the Nx/Torchx combo, because axon_onnx didn’t work with the last v0.7.1 of Nx. The dll loading seems to have changed since then.

Thanks for mentioning Ortex. I was totally unaware of that project. I made a first version of this project a year ago. Back then axon_onnx was the way to go. That first version was created as a webapplication, so I didn’t have the requirement of having to compile it for Windows.

WSL could be an option, but makes distribution and installation a bit harder. Although I’m pleasantly surprised how easy it is to set up on Windows 11 (hadn’t tried that before).
But I think you’re right to think in that direction, given the support for all the ML libraries is better on the unix systems.

4

I have a follow-up question regarding Ortex. Does Ortex bring it’s own accelerator, or does it still depend the Nx.Backend for acceleration? If Ortex would alleviate the need for a compiled backend, that would simplify things a lot.

5

It brings its own backend and you need to load models specific to it.

1 Like