ollran

ollran

Running Elixir with limited heap size

Hi,

Is it possible to somehow limit the memory usage of the Erlang virtual machine? I’m on a shared server and I don’t have root access.

For example, when I want to run Java I cannot use the default settings:

$ java
Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes

I have to start the JVM with limited heap size:

$ java -XX:MaxHeapSize=512m -XX:CompressedClassSpaceSize=64m -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

$ java -XX:MaxHeapSize=512m -XX:CompressedClassSpaceSize=64m HelloWorld
Hello, World

I am using the new release feature that came in Elixir 1.9. When I try to run my Elixir application I get this error:

$ _build/prod/rel/webapp/bin/webapp start
2019-07-02 15:53:49.850859
    args: [load_failed,"Failed to load NIF library: '/var/www/_build/prod/rel/webapp/lib/asn1- 5.0.9/priv/lib/asn1rt_nif.so: failed to map segment from shared object: Cannot allocate memory'"]
format: "Unable to load asn1 nif library. Failed with error:~n\"~p, ~s\"~n"
label: {error_logger,error_msg}
2019-07-02 15:53:49.852625 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.976.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,asn1rt_nif},[{init,run_on_load_handlers,0,[]},{kernel,init,1,[{file,"kernel.erl"},{line,187}]},{supervisor,init,1,[{file,"supervisor.erl"},{line,295}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[kernel_sup,<0.950.0>]},{message_queue_len,0},{messages,[]},{links,[<0.952.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,265}],[]]}
2019-07-02 15:53:49.852655 supervisor_report   #{label=>{supervisor,start_error},report=>[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,asn1rt_nif}},{offender,[{pid,undefined},{id,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
2019-07-02 15:53:50.858601 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.949.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[<0.948.0>]},{message_queue_len,1},{messages,[{'EXIT',<0.950.0>,normal}]},{links,[<0.948.0>,<0.947.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,193}],[]]}
2019-07-02 15:53:50.861379 std_info            #{label=>{application_controller,exit},report=>[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...done

Is it possible to limit the heap size in Elixir/Erlang?

Marked As Solved

alvises

alvises

To limit VM heap size I’ve found this: [erlang-questions] Max heap size

It says to pass the options +MMsco true +Musac false +MMscs 60 (You find these options here erts_alloc — OTP 29.0.2 (erts 17.0.2)) where 60 is the limit in MB.

I’ve tried it and when the VM reached the limit it crashes…

 eheap_alloc: Cannot allocate 600904 bytes of memory (of type "heap")

@ollran what does it happen to the JVM when it reaches the memory limit?

Also Liked

garazdawi

garazdawi

Erlang Core Team

You may want to try +Meamin. This will disable all the erts memory allocators and fall back to malloc which can sometimes use less memory.

NobbZ

NobbZ

There should be plenty of options available to deal with that, but it seems as if your problem is in a NIF. It won’t be restricted by the VMs Heap.

NobbZ

NobbZ

Its not at all… Consider it an external program that runs side by side to your erlang program and both are talking via a defined interface. The BEAM can’t control anything about that program except for its lifecycle… Start and stop…

But how restricted is your memory? I had erlang applications starting with less than 100 MB available, I had myself a DO 5$ droplet where the application never used more than 150 MB peak…

Also, as the problem seems to be asn1 related, do you really need it or can you strip that dependency?

Where Next?

Popular in Questions Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

Other popular topics Top

malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New

We're in Beta

About us Mission Statement