I had similar issues and would be interested to find a solution, but I didn’t find such flag.

In fact, figuring out which processes were even responsible for allocating memory was difficult on BEAM.

I ended up sampling the state of the system every 30s or so and be warned if we detect a process that exceeds something like 15 MB of memory.

The code I’m using is something like:

defmodule Infra.Bloat.Find do                                                                                                                                            
  @info_keys [                                                                                                                                                           
    :current_function,                                                                                                                                                   
    :initial_call,                                                                                                                                                       
    :status,                                                                                                                                                             
    :message_queue_len,                                                                                                                                                  
    :links,                                                                                                                                                              
    :dictionary,                                                                                                                                                         
    :trap_exit,                                                                                                                                                          
    :error_handler,                                                                                                                                                      
    :priority,                                                                                                                                                           
    :group_leader,                                                                                                                                                       
    :total_heap_size,                                                                                                                                                    
    :heap_size,                                                                                                                                                          
    :stack_size,                                                                                                                                                         
    :reductions,                                                                                                                                                         
    :garbage_collection,                                                                                                                                                 
    :suspending,                                                                                                                                                         
    :memory                                                                                                                                                              
  ]                                                                                                                                                                      
                                                                                                                                                                         
  @fifteen_megs 1024 * 1024 * 15                                                                                                                                         
                                                                                                                                                                         
  def processes do                                                                                                                                                       
    Process.list()                                                                                                                                                       
    |> Enum.map(fn pid ->                                                                                                                                                
      Process.info(pid, @info_keys)                                                                                                                                      
    end)                                                                                                                                                                 
    |> Enum.filter(&(&1[:memory] && &1[:memory] >= @fifteen_megs
  end           
end

I run Infra.Bloat.Find every 30s and log if something was found. Then, I look at the logs and if something showed up I fix the memory leak.

The usual suspects, according to my experience are:

• long running GenServers that do very little work, preventing GC from kicking in. Suspending these processes when idle does the job here.
• Absinthe GraphQL resolvers that were written in naive way, basically exploding the returned size of the payload and / or doing a lot of N+1 queries. Data loader / and / or adding pagination to these helps a lot.
• processing uploaded files, and/or processing JSON. I ended up writing a pretty shitty streaming subset of JSON parser because of the API I’m using tends to return super large arrays of things in JSON and this was crashing the system easily.

With GraphQL resolvers or just a web requests that are crashing your pod, you need to know that BEAM is able to allocate a lot of memory very very fast, so you won’t catch all of such spikes. Some will kill your pod, others won’t but will go unnoticed, so I ended up just running the sampling code constantly on production and making sure nothing new exceeds the arbitrary memory limit I’ve set up. This seems to work really well at scale and I am able to detect memory leak issues before they are the problem.

As BEAM is process-oriented you can set a per-process memory limit. By default, no process has a memory limit and they are all allowed to consume as much as they want.

You can change that default though. E.g. to change the limit to a reasonable 10MB if you’re launching your instance from a shell script add the export ELIXIR_ERL_OPTIONS:

export ELIXIR_ERL_OPTIONS="+hmax 10000000"
iex -S mix

Or if you’re using a release put that into your rel/vm.args.eex

+hmax 10000000

In addition to that, it’s possible to change the max process memory on a per-process level. So you can go with a default limit of 10MB per process as above but then increase that for certain “important” processes, or reduce it for less important processes.

Also, you could stay without a global default limit but set a per request limit from an embedded plug. For E.g. adding this to the beginning of your endpoint definition would set all phoenix request memory limits to 1MB:

defmodule BuzzWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :buzz

  # Setting request memory limit
  plug :set_memory_limit
  defp set_memory_limit(conn, _opts) do
    :erlang.process_flag(:max_heap_size, 1_000_000)
    conn
  end

If you’re using liveview this does not affect the liveview processes as they don’t run through these endpoint plugs. In that case, you could do it from the mount callback or similar.

Hope this helps.
Cheers!

A book suggestion on this topic is Erlang in Anger. It’s free and is one of the best resources that I read to understand what things can go wrong and how to diagnose them.

That said, one thing I might try in a high-memory situation is to force GC globally to see if the memory drops. I do this with Process.list() |> Enum.each(& :erlang.garbage_collect/1). If you do see a large memory drop, then you may have a “memory leak”. But it may not be a memory leak like you’re used to where there’s unfreed memory, but rather processes that don’t have the opportunity to GC due to the lifecycle of the BEAM. One thing you can do to give the opportunity to GC more frequently is to set the VM flag of -env ERL_FULLSWEEP_AFTER 20 in your VM.args file.

I set the above flag in every Elixir app I build now. I have not seen any negative side effects from it (can increase CPU usage, but I did not see a noticeable change) and the benefits can be significant in some long-lived processes.

I wrote an old post about how I diagnosed this in one of my services. The post itself is irrelevant because Phoenix Channels hibernate by default now, but the content itself is still relevant.

edit: 1 more question. What does your memory usage look like? There are several types of memory in the BEAM (process, binary, atom, etc) and that can determine the specific issue. I am not familiar with LiveDashboard as I have used observer_cli in my projects, but I imagine that’s one of the main breakdowns it provides.