I am trying to add support for Nerves in Vix (an Image Processing library) and facing a blocker related to cross-compilation.
Vix uses NIF to communicate with libvips library. It uses libvips introspection and elixir macros to generate code and documentation for all the libvips operations. Since macro expansion happen during elixir code compilation, Vix loads NIF during the code compilation.
When the compilation target and runtime target is same, it just loads the NIF
elixir_make compiled. But in case of Nerves they both differ. For example in my case, I am trying to create firmware in macos targeting RPi3. During the elixir code compilation it tries to load the NIF which is created for RPi3 and fails. In other words, how to handle native dependencies during cross compilation.
Anyone faced similar issue, or have any workarounds?
Few possible workarounds I can think of
- just create firmware on the target itself. i.e. create the firmware (compile) in the RPi3 itself.
- slightly ugly, but we could hack around
priv/* directory by replacing NIF files to match target after code compilation
- dynamically load different NIF file (
:erlang.load_nif(nif_path, 0)) based on current architecture. Again, this is ugly since I have to juggle between multiple NIF files for different targets.
If I understand correctly then I think this might help you:
Looking at your project, I don’t think this is the issue, but let me start with a general statement since this comes up periodically.
elixir_make projects, there are few important things to make NIFs work well with Nerves:
- Use macros like
CFLAGS instead of hardcoding calls to
gcc. A lot of projects already do this, so sometimes this is really easy. See Environment variables — nerves v1.10.4.
- Put outputs in
$(MIX_APP_PATH)/priv to keep host and target binaries separated. Nerves uses Mix’s
MIX_TARGET feature which will create a new build directory for each target you compile for.
- Use Mix’s
$(ERL_EI_LIBDIR) instead of calling Erlang in the Makefile to figure out where Erlang header files and libraries are.
The standard procedure is to copy/paste a
Makefile from another Elixir project to make it work with Nerves. For an example, see circuits_spi’s Makefile for a NIF example.
Nerves uses Buildroot to handle complicated C/C++ libraries. I took a look at
libvips, and they made a few design decisions that I honestly don’t find enjoyable to work around. If I were required to use it, I’d do what was suggested and make a custom Nerves system to build
libvips. The Buildroot package is at Buildroot libvips. Once that’s done, the Elixir Vix library would need to properly detect that it exists. That may work.
This is a long way of saying this:
- Give up trying to manually cross-compile
libvips for Nerves
- If compiling with Nerves (checking
$(CROSSCOMPILE) in the
Makefile is probably easiest), make sure that the mode is
pkg-config call should look in the right place.
- If crosscompiling and not using a platform-provided libvips or it can’t be found, tell the user to build a custom Nerves system and
BR2_PACKAGE_LIBVIPS=y to their
nerves_defconfig. While at it, the user may want to also enable other libraries since libvips optionally works with so many.
Unfortunately, this will make Vix less interesting to Nerves users since it won’t work with the official Nerves systems, but it should work for anyone who needs it.
Hey @fhunleth thanks for the detailed response.
As you said, first 3 general points related to NIF are already covered. Noted your other points. I don’t know much about Nerves and Buildroot, but I thought I just have to fix the elixir code compilation part, given I can already cross-compile NIF to the target architecture (RPi3) successfully. I guess I have to learn more about buildroot.