mikl
Mix deps.get memory explosion when doing cross-platform Docker build
I’m trying to do a cross-platform build for a Phoenix app, so on my ARM macOS, I’m running
docker buildx build --platform linux/amd64 [more, less relevant options] .
The relevant part of the Dockerfile contains:
FROM hexpm/elixir:1.15.2-erlang-26.0.2-ubuntu-focal-20230126 AS build-stage
# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
&& apt-get clean && rm -f /var/lib/apt/lists/*_*
# prepare build dir
WORKDIR /app
# install hex + rebar
RUN mix local.hex --force && \
mix local.rebar --force
# set build ENV
ENV MIX_ENV="prod"
# install mix dependencies
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
When it reaches the RUN mix deps.get line, it blows up after a brief moment with BEAM running out of memory:
=> ERROR [build-stage 6/17] RUN mix deps.get --only prod 3.5s
------
> [build-stage 6/17] RUN mix deps.get --only prod:
3.040 no next heap size found: 2305842975251512930, offset 0
3.045
3.045 Crash dump is being written to: erl_crash.dump...done
3.483 qemu: uncaught target signal 6 (Aborted) - core dumped
3.493 Aborted
------
Dockerfile:22
--------------------
20 | # install mix dependencies
21 | COPY mix.exs mix.lock ./
22 | >>> RUN mix deps.get --only $MIX_ENV
23 | RUN mkdir config
24 |
--------------------
Things I’ve already tried:
- Omitting
--platform linux/amd64makes the memory explosion go away. But then the binaries built will be Aarch64 which won’t run on an x64 Linux server. - Skipping
--only prodmakes no difference, getting the deps still has a memory explosion.
I understand this is not a common thing to do, but if someone has an idea on how to solve this, I’d be grateful.
Marked As Solved
pdgonzalez872
hi @mikl! I was reading a thread a while back that may help you:
- Started here: Pros and Cons of Apple silicon for Elixir dev? - #5 by josevalim, which linked this PR: Update Elixir and Erlang versions (#1939) · livebook-dev/livebook@6256b66 · GitHub
- which then in turn linked: https://github.com/erlang/otp/pull/6340
- and finally, an example of the usage: fix qemu arm64 builds by ruslandoga · Pull Request #2879 · plausible/analytics · GitHub
Could you please try that out and report back? Thanks!
Also Liked
mikl
Wow, that’s quite the rabbit hole.
I’m very happy to report that adding ENV ERL_FLAGS="+JPperf true" to the build-section of my Dockerfile fixes this.
Thanks, @pdgonzalez872.
(To anyone else needing this workaround, if you consult the Erlang docs for that flag, this is indeed recommended as a workaround for Qemu-specific problems, not something you should have set on your server).
fteschke
It is probably sufficient to use +JMsingle instead of +JPperf
mix release — Mix v1.20.2
However, when building such images on your machine, those technologies use emulation which may not interplay well with Erlang VM’s JIT (just-in time) compiler. To address this, you can set this environment variable on your build stage:
ENV ERL_AFLAGS “+JMsingle true”
erl — OTP 29.0.2 (erts 17.0.2)
+JMsingle: This flag is automatically enabled by the+JPperfflag.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








