:erlang.memory(:total) shows 54877864,
VIRT shows 4G.
When I do ulimit -Sv 1048576 app just fails to start with error:
erts_mmap: Failed to create super carrier of size 1024 MB.
+MMscs 64
+MMsco true
+MMscrpm true
+Musac false
in vm.args does not reduce VIRT memory amount.
Searched this forum - found nothing of use.
Are there are any methods to reduce it?
Does that reply by one of Erlang’s maintainers help?
App is not in a container.
In vm.args:
+P 1024 - max processes 1024
+Q 1024 - max ports 1024
Both does not help.
Still the same 4G.
OS: Debian 10.
Elixir 1.15.
Erlang 26.0.2.
Did you read the BEAM VM option he linked to? This is not about containers, it’s an option that should work regardless of where the app is started.
Hm, seems there is more that can be done but I am not aware of it, sorry.
Wonder if @garazdawi would be willing to help further.
I would use recon_alloc — recon v2.5.3 to get a view of what is using memory. Especially recon_alloc:fragmentation/1 can be helpful to look at.
If you can provide a recon_alloc snapshot I can see if I can figure out what is using all the extra virtual memory.
The statistics for the Erlang allocators do not indicate that you are using a lot of virtual memory. What does :erlang.system_info({:allocator,:erts_mmap}) return?
[
default_mmap: [options: [scs: 0], os: [sizes: [used: 29888512]]],
literal_mmap: [
options: [scs: 1073676288, sco: true, scrpm: false, scrfsd: 1024],
supercarrier: [
sizes: [
total: 1073741824,
total_sa: 4096000,
total_sua: 0,
used: 4161536,
used_sa: 4096000,
used_sua: 0
],
free_segs: [
used: 0,
max: 0,
allocated: 0,
reserved: 1024,
used_sa: 0,
used_sua: 0
]
]
]
]
Summary from /proc/16025/smaps where 16025 - beam pid.
* Mem usage for PID 16025
-- Size: 4172764 kB
-- Rss: 135352 kB
-- Pss: 113538 kB
Shared Clean 5440 kB
Shared Dirty 34504 kB
Private 95412 kB
Some very strange region
VmFlags: rd wr mr mw me ac sd
7f659a834000-7f65da43b000 ---p 00000000 00:00 0
Size: 1044508 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Rss: 0 kB
Pss: 0 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 0 kB
Anonymous: 0 kB
LazyFree: 0 kB
AnonHugePages: 0 kB
ShmemPmdMapped: 0 kB
Shared_Hugetlb: 0 kB
Private_Hugetlb: 0 kB
Swap: 0 kB
SwapPss: 0 kB
Locked: 0 kB
THPeligible: 1
ProtectionKey: 0
And many empty regions with the size 65504 kB
VmFlags: rd wr mr mw me nr sd
7f6530008000-7f6534000000 ---p 00000000 00:00 0
Size: 65504 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Rss: 0 kB
Pss: 0 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 0 kB
Anonymous: 0 kB
LazyFree: 0 kB
AnonHugePages: 0 kB
ShmemPmdMapped: 0 kB
Shared_Hugetlb: 0 kB
Private_Hugetlb: 0 kB
Swap: 0 kB
SwapPss: 0 kB
Locked: 0 kB
THPeligible: 1
ProtectionKey: 0
My script for summary:
#!/bin/bash
pid=$1
echo "* Mem usage for PID $pid"
sz=$(awk 'BEGIN {i=0} /^Size/ {i = i + $2} END {print i}' /proc/$pid/smaps)
rss=$(awk 'BEGIN {i=0} /^Rss/ {i = i + $2} END {print i}' /proc/$pid/smaps)
pss=$(awk 'BEGIN {i=0} /^Pss/ {i = i + $2 + 0.5} END {print i}' /proc/$pid/smaps)
sc=$(awk 'BEGIN {i=0} /^Shared_Clean/ {i = i + $2} END {print i}' /proc/$pid/smaps)
sd=$(awk 'BEGIN {i=0} /^Shared_Dirty/ {i = i + $2} END {print i}' /proc/$pid/smaps)
pc=$(awk 'BEGIN {i=0} /^Private_Clean/ {i = i + $2} END {print i}' /proc/$pid/smaps)
pd=$(awk 'BEGIN {i=0} /^Private_Dirty/ {i = i + $2} END {print i}' /proc/$pid/smaps)
echo "-- Size: $sz kB"
echo "-- Rss: $rss kB"
echo "-- Pss: $pss kB"
echo "Shared Clean $sc kB"
echo "Shared Dirty $sd kB"
echo "Private $(($pd + $pc)) kB"
Do you have nifs that are leaking memory?
Don’t have any NIFs.
:recon_alloc.fragmentation(:max)
[
{{:ll_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.6853810185971467,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 8264752,
mbcs_carriers_size: 12058624
]},
{{:eheap_alloc, 6},
[
sbcs_usage: 0.9943773109494275,
mbcs_usage: 0.3585852420691288,
sbcs_block_size: 4268472,
sbcs_carriers_size: 4292608,
mbcs_block_size: 1551016,
mbcs_carriers_size: 4325376
]},
{{:ll_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5442423502604167,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 2140048,
mbcs_carriers_size: 3932160
]},
{{:sl_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5040318080357142,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 1734192,
mbcs_carriers_size: 3440640
]},
{{:eheap_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3121984145220588,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 695648,
mbcs_carriers_size: 2228224
]},
{{:sl_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.23021865472560976,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 309296,
mbcs_carriers_size: 1343488
]},
{{:eheap_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3093483664772727,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 446016,
mbcs_carriers_size: 1441792
]},
{{:sl_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2669647961128049,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 358664,
mbcs_carriers_size: 1343488
]},
{{:eheap_alloc, 15},
[
sbcs_usage: 0.9980070153061225,
mbcs_usage: 0.3265380859375,
sbcs_block_size: 1201824,
sbcs_carriers_size: 1204224,
mbcs_block_size: 470800,
mbcs_carriers_size: 1441792
]},
{{:sl_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2813155011432927,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 377944,
mbcs_carriers_size: 1343488
]},
{{:sl_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.31162466653963417,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 418664,
mbcs_carriers_size: 1343488
]},
{{:eheap_alloc, 2},
[
sbcs_usage: 0.9980070153061225,
mbcs_usage: 0.3696233575994318,
sbcs_block_size: 600912,
sbcs_carriers_size: 602112,
mbcs_block_size: 532920,
mbcs_carriers_size: 1441792
]},
{{:sl_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3430056688262195,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 460824,
mbcs_carriers_size: 1343488
]},
{{:sl_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3623404153963415,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 486800,
mbcs_carriers_size: 1343488
]},
{{:sl_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.41235947027439024,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 554000,
mbcs_carriers_size: 1343488
]},
{{:sl_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4502667682926829,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 604928,
mbcs_carriers_size: 1343488
]},
{{:binary_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.6695237379807693,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 1426032,
mbcs_carriers_size: 2129920
]},
{{:eheap_alloc, 4},
[
sbcs_usage: 0.9976207386363637,
mbcs_usage: 0.5206132368607954,
sbcs_block_size: 3146416,
sbcs_carriers_size: 3153920,
mbcs_block_size: 750616,
mbcs_carriers_size: 1441792
]},
{{:ll_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.6317618233816964,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 1159288,
mbcs_carriers_size: 1835008
]},
{{:std_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5447313262195121,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 731840,
mbcs_carriers_size: 1343488
]},
{{:temp_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5175442165798612,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 610520,
mbcs_carriers_size: 1179648
]},
{{:ll_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 524288
]},
{{:temp_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5556301540798612,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 655448,
mbcs_carriers_size: 1179648
]},
{{:ll_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.010009765625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 5248,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.03369140625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 17664,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.036895751953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 19344,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.06475830078125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 33952,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0824432373046875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 43224,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.090545654296875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 47472,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.167388916015625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 87760,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1899566650390625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 99592,
mbcs_carriers_size: 524288
]},
{{:ll_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.7914515904017857,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 1452320,
mbcs_carriers_size: 1835008
]},
{{:ll_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3092803955078125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 162152,
mbcs_carriers_size: 524288
]},
{{:ets_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.7428306021341463,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 997984,
mbcs_carriers_size: 1343488
]},
{{:ll_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.34747314453125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 182176,
mbcs_carriers_size: 524288
]},
{{:binary_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1123318142361111,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 33128,
mbcs_carriers_size: 294912
]},
{{:binary_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.11981879340277778,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 35336,
mbcs_carriers_size: 294912
]},
{{:fix_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.12291124131944445,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 36248,
mbcs_carriers_size: 294912
]},
{{:binary_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1282009548611111,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 37808,
mbcs_carriers_size: 294912
]},
{{:binary_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1322970920138889,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 39016,
mbcs_carriers_size: 294912
]},
{{:sl_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1406521267361111,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 41480,
mbcs_carriers_size: 294912
]},
{{:std_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.15364583333333334,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 45312,
mbcs_carriers_size: 294912
]},
{{:fix_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1587456597222222,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 46816,
mbcs_carriers_size: 294912
]},
{{:eheap_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.37030029296875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 145608,
mbcs_carriers_size: 393216
]},
{{:fix_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.16959635416666666,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 50016,
mbcs_carriers_size: 294912
]},
{{:std_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1769748263888889,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 52192,
mbcs_carriers_size: 294912
]},
{{:fix_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.185791015625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 54792,
mbcs_carriers_size: 294912
]},
{{:eheap_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4459228515625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 175344,
mbcs_carriers_size: 393216
]},
{{:fix_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2789442274305556,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 82264,
mbcs_carriers_size: 294912
]},
{{:ll_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.633697509765625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 332240,
mbcs_carriers_size: 524288
]},
{{:binary_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.3900824652777778,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 115040,
mbcs_carriers_size: 294912
]},
{{:eheap_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5630086263020834,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 221384,
mbcs_carriers_size: 393216
]},
{{:binary_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4283311631944444,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 126320,
mbcs_carriers_size: 294912
]},
{{:std_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4959038628472222,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 146248,
mbcs_carriers_size: 294912
]},
{{:temp_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 4.2724609375e-4,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 56,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 4.2724609375e-4,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 56,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.05029296875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 6592,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.05535888671875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 7256,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0618896484375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 8112,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.07073974609375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 9272,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.079345703125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 10400,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.08050537109375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 10552,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.08184814453125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 10728,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0823974609375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 10800,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.08612060546875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 11288,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.70123291015625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 275736,
mbcs_carriers_size: 393216
]},
{{:temp_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.11688232421875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 15320,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.1348876953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 17680,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.14324951171875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 18776,
mbcs_carriers_size: 131072
]},
{{:sl_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.62109375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 183168,
mbcs_carriers_size: 294912
]},
{{:temp_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.15008544921875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 19672,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.15130615234375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 19832,
mbcs_carriers_size: 131072
]},
{{:sl_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.6443956163194444,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 190040,
mbcs_carriers_size: 294912
]},
{{:temp_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2113037109375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 27696,
mbcs_carriers_size: 131072
]},
{{:sl_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.6630859375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 195552,
mbcs_carriers_size: 294912
]},
{{:temp_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.28680419921875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 37592,
mbcs_carriers_size: 131072
]},
{{:eheap_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.7637939453125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 300336,
mbcs_carriers_size: 393216
]},
{{:eheap_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.30157470703125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 39528,
mbcs_carriers_size: 131072
]},
{{:sl_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.7358940972222222,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 217024,
mbcs_carriers_size: 294912
]},
{{:temp_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.50006103515625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 65544,
mbcs_carriers_size: 131072
]},
{{:temp_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5096435546875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 66800,
mbcs_carriers_size: 131072
]},
{{:sl_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.8407931857638888,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 247960,
mbcs_carriers_size: 294912
]},
{{:sl_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 4},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.001953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 64,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 7},
[
sbcs_usage: 1.0,
mbcs_usage: 0.001953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 64,
mbcs_carriers_size: 32768
]},
{{:sl_alloc, 8},
[
sbcs_usage: 1.0,
mbcs_usage: 0.001953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 64,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.001953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 64,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.109130859375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 3576,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 9},
[
sbcs_usage: 1.0,
mbcs_usage: 0.220703125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 7232,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2275390625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 7456,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.2275390625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 7456,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.26171875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 8576,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 10},
[
sbcs_usage: 1.0,
mbcs_usage: 0.271728515625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 8904,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.302734375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 9920,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.30712890625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 10064,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.350341796875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 11480,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.384765625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 12608,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4072265625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 13344,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4189453125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 13728,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.437744140625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 14344,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.44921875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 14720,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.454833984375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 14904,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 11},
[
sbcs_usage: 1.0,
mbcs_usage: 0.48046875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 15744,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 13},
[
sbcs_usage: 1.0,
mbcs_usage: 0.4853515625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 15904,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 1},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5078125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 16640,
mbcs_carriers_size: 32768
]},
{{:std_alloc, 16},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5107421875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 16736,
mbcs_carriers_size: 32768
]},
{{:driver_alloc, 6},
[
sbcs_usage: 1.0,
mbcs_usage: 0.544677734375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 17848,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 0.5830078125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 19104,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 14},
[
sbcs_usage: 1.0,
mbcs_usage: 0.62451171875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 20464,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 5},
[
sbcs_usage: 1.0,
mbcs_usage: 0.71337890625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 23376,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 15},
[
sbcs_usage: 1.0,
mbcs_usage: 0.7578125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 24832,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.758056640625,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 24840,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 12},
[
sbcs_usage: 1.0,
mbcs_usage: 0.833984375,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 27328,
mbcs_carriers_size: 32768
]},
{{:binary_alloc, 3},
[
sbcs_usage: 1.0,
mbcs_usage: 0.8388671875,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 27488,
mbcs_carriers_size: 32768
]},
{{:fix_alloc, 2},
[
sbcs_usage: 1.0,
mbcs_usage: 0.92626953125,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 30352,
mbcs_carriers_size: 32768
]},
{{:ets_alloc, 0},
[
sbcs_usage: 1.0,
mbcs_usage: 1.0,
sbcs_block_size: 0,
sbcs_carriers_size: 0,
mbcs_block_size: 0,
mbcs_carriers_size: 0
]}
]
I may have figured it out. It is the malloc arenas that are using all the extra virtual memory.
If I set MALLOC_ARENA_MAX=1 then the virtual memory usage on my machine drops by a lot.
It is this allocation that causes malloc to create an arena on each thread.
That being said, I’m not sure we will do anything about this. Virtual memory does not cost anything, except that limits placed on it will have to be adjusted as you have noticed.
Oh my! This is it!
It’s magic!
Thank you!
export MALLOC_ARENA_MAX=1
+
in vm.args:
+Q 1024
+P 1024
+MIscs 128
Dropped it to 332.4MB of VIRT memory.
Also thank you Dimitar 
© Copyright Elixir Forum | Terms | Privacy & Cookies
