PatchELF instead of Docker? :-)

Chat / Discussions
,
1

Releases is a fantastic new feature in Elixir 1.9, but it comes with one important caveat:

Once a release is assembled, it can be packaged and deployed to a target, as long as the target runs on the same operating system (OS) distribution and version as the machine running the mix release command.

If your dev machine is not running the same Linux distro and version than your server then chances are that you won’t be able to start your service because the packaged ERTS will probably be dynamically linked with different versions of libc, ncurses, or openssl. The solution I believe most people are currently using is to create a Docker container to build the release and then generate an image with the app or just match the target OS.

There were some recent discussions about the possibility of generating universal statically linked fat binaries, but here there is maybe an (hacky) alternative: just modify the ELF interpreter location and add a runpath with the same library versions used by your dev machine to all the Erlang distribution binaries included in the release.

Here is how one can achieve this with NixOS’s patchelf; let’s say we already have the release package (built in Arch Linux) in our target machine running Debian Buster:

$ cd myapp/erts-10.5/bin
$ ./beam.smp
./beam.smp: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./beam.smp)
$ file beam.smp
beam.smp: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=eb7be73353afbb0bc4962b81bd427aed7ce66dcd, for GNU/Linux 3.2.0, stripped
$ ldd -v beam.smp
./beam.smp: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./beam.smp)
[...]
        Version information:                  
        ./beam.smp:                                                                                       
[...]
                libm.so.6 (GLIBC_2.29) => not found                                                       
[...]

So now we need to provide the glibc package used by our local dev machine to the target system, extract it somewhere, and finally modify the ERTS binaries to look for these libraries (I will skip the scp or curl part):

$ cd
$ tar xf glibc-2.29.tar
$ patchelf --debug --set-interpreter /home/user/glibc-2.29/lib64/ld-linux-x86-64.so.2 --set-rpath /home/user/glibc-2.29/lib/x86_64-linux-gnu/ myapp/erts-10.5/bin/beam.smp

(we probably want to do this with the rest of the ELF files: ct_run, dialyzer, dyn_erl, epmd, erl_child_setup, erlc, erlexec, escript, heart, inet_gethost, run_erl, to_erl, typer)

At this point BEAM should be able to run and your minimal Elixir application should be able to start.

Don’t do this at work! :joy:

6 Likes
2

You do not need to patch ELF executables, you can use LD_LIBRARY_PATH for the same effect.

1 Like
3

Unfortunately using LD_LIBRARY_PATH doesn’t work in this case as the ld-linux must match with the libc version (I believe you would just get a segmentation fault), but please let me know if you find some way to do it without patching the binaries.

4

I was thinking that it should be possible to do all this automatically (i.e. check direct and indirect dependencies, check ld interpreter, copy everything into the release folder, patch every ELF binary with the relative path where the libraries and interpreter were copied). That would make the release compatible between Linux distributions making deployments easier in some situations.

It seems this is how NixOS does it for non-open-source applications: https://sandervanderburg.blogspot.com/2018/10/auto-patching-prebuilt-binary-software.html

5

Here I wrote a script that performs all this process automatically: ducktape.sh

It was tested successfully between Arch Linux (dev box) and Debian 9 (server) :smile:

This tool needs patchelf (v0.9) only in your local machine, and it generates a script called patchpath.sh in your release’s root that you must run in your target host to hardcode the absolute path to the ld-linux file.

6 Likes