CPU usage have been increasing more than 100% to 130% for Elixir BEAM

Hi all, we have been using the elixir / phoenix app in production for few months. Till date, we have hit around 50 concurrent users connecting to the channels at any given point of time. The app worked fine with out any issues.

When ever the connections increase to around 150- 200, the CPU usage have been increasing more than 100% to 130%. How can we trace the issue. We are running two nodes in AWS EC2 (each C4 Large - 2VCPUs, 8 ECU). The memory usage is not exceeding 30%

More about the app: This app is a multi-player game server. The users/players will keep connected at least for 10-15 minutes. They play with other players. So, the channels broadcast the messages to other connected players. Each game runs in a process (or few child processes).

I would primarily explore what processes are the busy ones. I recommend the wonderful (and free) “Erlang in Anger” book and the accompanying tool - recon. In particular the :recon.proc_window(:reductions, n, time) function to gather CPU usage data (a reduction is more-or-less equivalent to a function call) over a sliding time window and print n largest users. This will be the good starting point to finding the culprit.

5 Likes