Building Elixir/Erlang linux/amd64 application image on Apple Silicon

I have a multi stage docker file and building the node assets works fine with both --platform linux/arm64 and linux/amd64. I can make the elixir/erlang image work with --platform linux/arm64, but when I try linux/amd64 qemu blows up:

FROM --platform=linux/amd64 hexpm/elixir:1.12.3-erlang-24.1.5-ubuntu-focal-20210325 AS deps

ARG MIX_ENV=prod
ARG HEX_MIRROR_URL=https://repo.hex.pm

RUN /usr/local/bin/mix local.hex --force && \
    /usr/local/bin/mix local.rebar --force

RUN apt-get -qq update && apt-get install -y \
    git \
    unzip \
    build-essential \
    wget \
    sudo

WORKDIR /opt/app

COPY config/ ./config
COPY mix.exs mix.lock /opt/app/

RUN mix deps.get --only $MIX_ENV

Results in

#12 [2/6] RUN /usr/local/bin/mix local.hex --force &&     /usr/local/bin/mix local.rebar --force
#12 sha256:690d11d4149aae5005ba9d4f6d1bfcb55259a6efa519068eac1913c8c5c6a1a1
#12 1.540 qemu: uncaught target signal 11 (Segmentation fault) - core dumped
#12 1.546 Segmentation fault
#12 ERROR: executor failed running [/bin/sh -c /usr/local/bin/mix local.hex --force &&     /usr/local/bin/mix local.rebar --force]: exit code: 139

Anyone else getting this problem? Or found a way around it?

Thanks for reading!

3 Likes

The hexpm 1.13.0-rc.1 image uploaded yesterday actually works!!

1.13.0-rc.1-erlang-23.2.7.4-ubuntu-focal-20210325

It seems the images with erlang 24.1 blow up

Some more digging here. It’s not OTP24 that fails, it is OTP24 on some distros:

OTP24 on Ubuntu Focal FAILS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-ubuntu-focal-20210325)
OTP24 on Ubuntu Bionic FAILS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-ubuntu-bionic-20210325)
OTP24 on Ubuntu Xenial WORKS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-ubuntu-xenial-20210114)
OTP24 on Debian Stretch WORKS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-debian-stretch-20210902)
OTP24 on Debian Bullseye FAILS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-debian-bullseye-20210902)
OTP24 on Alpine 3.14.2 FAILS (hexpm/elixir:1.13.0-rc.1-erlang-24.1.5-alpine-3.14.2)

4 Likes

The crash happens when JIT is enabled. Disabling the JIT, using pre-JIT Erlang (23.x or older), or using an old OS that doesn’t have a version of gcc recent enough to support C++17 (required to compile the JIT) will work around it.

6 Likes

Aha! Thank you for clearing this up. Is this something that will never work due to compatibility issues you think, or is it possible, but a bug with qemu?

I wish I knew! I also have that problem and I don’t know how to debug it :sweat_smile:

1 Like

Is there any permanent solution to this?

Not really.

I raised an issue on the docker for mac github, but it was closed and deferred to qemu.

There is a new qemu version out, but it hasn’t been picked up by docker yet (afaict)

4 Likes

It’s been handled poorly. Hopefully this time the issue might stay open so at least we learn where the culprit is or how to work around it Yet Another QEMU Segmentation Fault Issue · Issue #6261 · docker/for-mac · GitHub

1 Like

I’ve had the same issue today and trying with the Ubuntu image of Elixir 1.13 and OTP 25 worked for arm64 using QEMU using Github Actions. (image name: elixir:1.13.4-erlang-25.0.4-ubuntu-xenial-20210804)

1 Like

I also ran into this issue trying to build for linux/amd64 from an apple silicon macbook. I’m using mix phx.gen.release --docker and wanting to using Elixir 1.14 and OTP 25.

I managed to get it to work with ubuntu:xenial-20210804 which is quite old, released in 2016.

I tried the following distros to no success:

debian:bullseye-20230109-slim
debian:bullseye-20210902-slim
ubuntu:jammy-20230126
ubuntu:bionic-20230126

1 Like

This is due to a bug in qemu, see Erlang/OTP 25 JIT on AArch64 fails in user mode emulation (#1034) · Issues · QEMU / QEMU · GitLab.

Older images most likely work because they don’t provide a C++ compiler capable of compiling the JIT, making it fall back to the interpreter.

3 Likes