ollran

Running Elixir with limited heap size

Hi,

Is it possible to somehow limit the memory usage of the Erlang virtual machine? I’m on a shared server and I don’t have root access.

For example, when I want to run Java I cannot use the default settings:

$ java
Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes

I have to start the JVM with limited heap size:

$ java -XX:MaxHeapSize=512m -XX:CompressedClassSpaceSize=64m -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

$ java -XX:MaxHeapSize=512m -XX:CompressedClassSpaceSize=64m HelloWorld
Hello, World

I am using the new release feature that came in Elixir 1.9. When I try to run my Elixir application I get this error:

$ _build/prod/rel/webapp/bin/webapp start
2019-07-02 15:53:49.850859
    args: [load_failed,"Failed to load NIF library: '/var/www/_build/prod/rel/webapp/lib/asn1- 5.0.9/priv/lib/asn1rt_nif.so: failed to map segment from shared object: Cannot allocate memory'"]
format: "Unable to load asn1 nif library. Failed with error:~n\"~p, ~s\"~n"
label: {error_logger,error_msg}
2019-07-02 15:53:49.852625 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.976.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,asn1rt_nif},[{init,run_on_load_handlers,0,[]},{kernel,init,1,[{file,"kernel.erl"},{line,187}]},{supervisor,init,1,[{file,"supervisor.erl"},{line,295}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[kernel_sup,<0.950.0>]},{message_queue_len,0},{messages,[]},{links,[<0.952.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,265}],[]]}
2019-07-02 15:53:49.852655 supervisor_report   #{label=>{supervisor,start_error},report=>[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,asn1rt_nif}},{offender,[{pid,undefined},{id,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
2019-07-02 15:53:50.858601 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.949.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}},{ancestors,[<0.948.0>]},{message_queue_len,1},{messages,[{'EXIT',<0.950.0>,normal}]},{links,[<0.948.0>,<0.947.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,193}],[]]}
2019-07-02 15:53:50.861379 std_info            #{label=>{application_controller,exit},report=>[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,asn1rt_nif}}},{kernel,start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...done

Is it possible to limit the heap size in Elixir/Erlang?

14 comments

#beam #memory #mix-release

0 5266 14

2019-07-04 08:31:56 UTC

Marked As Solved

alvises

To limit VM heap size I’ve found this: [erlang-questions] Max heap size

It says to pass the options +MMsco true +Musac false +MMscs 60 (You find these options here erts_alloc — OTP 29.0.2 (erts 17.0.2)) where 60 is the limit in MB.

I’ve tried it and when the VM reached the limit it crashes…

 eheap_alloc: Cannot allocate 600904 bytes of memory (of type "heap")

@ollran what does it happen to the JVM when it reaches the memory limit?

Post #2

Also Liked

garazdawi

Erlang Core Team

You may want to try +Meamin. This will disable all the erts memory allocators and fall back to malloc which can sometimes use less memory.

Post #14

NobbZ

There should be plenty of options available to deal with that, but it seems as if your problem is in a NIF. It won’t be restricted by the VMs Heap.

Post #1

NobbZ

Its not at all… Consider it an external program that runs side by side to your erlang program and both are talking via a defined interface. The BEAM can’t control anything about that program except for its lifecycle… Start and stop…

But how restricted is your memory? I had erlang applications starting with less than 100 MB available, I had myself a DO 5$ droplet where the application never used more than 150 MB peak…

Also, as the problem seems to be asn1 related, do you really need it or can you strip that dependency?

Post #4