luechtdev

luechtdev

On my development VM (specs see below) - starting an elixir / erlang container takes up 2G of physical memory while running iex locally only takes up a few MB.
I only started an empty container with no modules loaded or code executed.
When more containers are started beyond the memory capabilities of the host, running Beam VM’s running in other (running) containers crash for the new instance to start.

Image of docker running with htop and system info (dev-vm)

observer_cli showing the memory usage

While memory usage is highly alarming on its own, other docker hosts are able to start the container with normal / reasonable memory consumption.

Production server running empty elixir container with normal memory usage

(Both instances running the same docker run -it elixir)

Other users could reproduce the problematic memory usage with the Ubuntu Desktop (kernel v5.15) but Ubuntu Server (kernel v6.4) is supposedly running just fine.
(Both running docker 24.0.2)

So far I tried:

  • running different versions of the image elixir:slim, elixir:alpine, …
  • limiting memory usage on docker → container crashes with errno 137 (out of memory);
  • checking for swap → swap is disabled on both devices;
  • running different versions of docker;

Any ideas on this problem would be apreciated since I’m unable to narrow down this problem to a Beam / Docker / OS / kernel level.

Kind Regards, Jannus

Showing Posts 1 to 10

LostKobrakai

LostKobrakai

I’d be curious how hexpm/elixir images fare. They’re imo the much more useful images over the elixir ones.

D4no0

D4no0

Is this true for a downgraded version of OTP, lets say OTP-25?

everte

everte

Just a quick test:

docker run --name elixirhexpm --rm -it hexpm/elixir:1.15.4-erlang-26.0.2-debian-buster-20230612-slim

Memory usage is 5MB; then I ran iex in the container. Memory usage is 2GB.

hexpm/elixir:1.15.4-erlang-25.3.2.4-debian-buster-20230612-slim (so OTP 25) has the exact same issue, but funny enough the memory usage is 1.5GB in stead of 2.0GB.

I had some screenshots as well in the (closed and duplicated topic): Elixir docker container very high memory usage

D4no0

D4no0

I ran this on a ARM instance server and it uses less than 5 MB, maybe there is a problem with hardware?

everte

everte

I think with only 5MB you forgot to run iex :).

As said: I cannot reproduce it everywhere. My desktop has this issue, but my linux vm/vps server cannot reproduce it. So there is definitely something else (docker, containerd, linux kernel, ??) that affects this issue. I have only tested on amd64.

But @luechtdev obviously has the same issue on some of his linux systems.

D4no0

D4no0

I’m not a linux master to understand how to correctly get the memory usage, however from my 8GB instance, it is using 1.6% with IEX opened, witch should be roughly 128 MB.

everte

everte

I check the memory usage with docker stats.

Anyway, when limiting the memory to 500M (which should be plenty for just an iex sesison) the container immediately crashes:

From dmesg on the host:

[49050.983989] Tasks state (memory values in pages):
[49050.983990] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[49050.983991] [  19403]     0 19403     1405        0    49152       96             0 bash
[49050.983993] [  19431]     0 19431   821098   127079  2166784   127776             0 beam.smp
[49050.983995] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-14bbfbecdada29e0e959b20b6d5e273609fcbb7d3440959d143a41b4ecebccdc.scope,mems_
allowed=0,oom_memcg=/system.slice/docker-14bbfbecdada29e0e959b20b6d5e273609fcbb7d3440959d143a41b4ecebccdc.scope,task_memcg=/system.slice/docker-14bbfbecdada29
e0e959b20b6d5e273609fcbb7d3440959d143a41b4ecebccdc.scope,task=beam.smp,pid=19431,uid=0
[49050.984007] Memory cgroup out of memory: Killed process 19431 (beam.smp) total-vm:3284392kB, anon-rss:508188kB, file-rss:128kB, shmem-rss:0kB, UID:0 pgtabl
es:2116kB oom_score_adj:0
D4no0

D4no0

Here you go:


8fda87bc816b   elixirhexpm                   0.00%     52.47MiB / 500MiB     10.49%    946B / 0B         0B / 0B           22

everte

everte

I created a new (arm) vm with fedora 38 and can reproduce it there as well (seems that fedora 38 seems to be one of the distros with new enough kernel/containerd/etc to reproduce).

docker run --name elixirhexpm --rm -it hexpm/elixir:1.15.4-erlang-26.0.2-debian-buster-20230612-slim iex

CONTAINER ID   NAME          CPU %     MEM USAGE / LIMIT    MEM %     NET I/O       BLOCK I/O   PIDS
b9f0d3e19106   elixirhexpm   0.00%     2.04GiB / 3.712GiB   54.95%    1.36kB / 0B   0B / 0B     21
[root@fedora-4gb-fsn1-2 ~]# uname -a
Linux fedora-4gb-fsn1-2 6.2.15-300.fc38.aarch64 #1 SMP PREEMPT_DYNAMIC Thu May 11 16:54:06 UTC 2023 aarch64 GNU/Linux
[root@fedora-4gb-fsn1-2 ~]# docker version
Client: Docker Engine - Community
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:37:12 2023
 OS/Arch:           linux/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:57 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
D4no0

D4no0

Yeah I’m running ubuntu:

Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-72-generic aarch64)

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
Onor.io
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
Trolleger
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
matt-savvy
Anyone here using Honeybadger? My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of Bandit.HTTPError...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews