Phoenix Channel Limit

Phoenix Forum Questions / Help
,
1

Hi, I have a setup of websockets using Phoenix Channels. I setup this configuration in Ubuntu:

sysctl -w fs.file-max=12000500
sysctl -w fs.nr_open=20000500
ulimit -n 20000000
sysctl -w net.ipv4.tcp_mem='10000000 10000000 10000000'
sysctl -w net.ipv4.tcp_rmem='1024 4096 16384'
sysctl -w net.ipv4.tcp_wmem='1024 4096 16384'
sysctl -w net.core.rmem_max=16384
sysctl -w net.core.wmem_max=16384

When I check the ulimit -n, I get 65535

But When I’m load testing the websockets app with Tsung it always gets stuck at 993 socket connections. I’m thinking that the issue can be a limit in the OS?

2

Sounds like you have a 1024 port limit somewhere. Try these places: https://www.cogini.com/blog/tuning-tcp-ports-for-your-elixir-app/

3

I have set the parameters as you mentioned. I’ve also followed the setup of The Road to 2 Million Socket connections. But I keep having the issue of being stuck al lower than 1000 sockets connections.

I have testes on a local vagrant machine, with the same configuration and I have no issues, what could be wrong? I have tried lightsail and Azure B1s, can these machines be lock to a hard limit?

@jakemorrison

4

I have scaled the VM, but the issue persists. If I check: ulimit -a, i get:

image

1 Like
5

Have you also checked the machine that is running Tsung? Perhaps the issue could be on that side.

6

I am running Tsung from my local machine, I have tested 5k concurrent users agains a vagrant machine, without issues, but when I test again a Cloud VM, it always get stuck a lower than 1k users.

I am targeting the direct IP of the cloud VM. I have used lightsail and azure, and the issue persists, seems like its a configuration on the cloud VM, but I have no way to debug what could it be.

@axelson

1 Like
7

OS: Ubuntu 18.04. These are the parameters I’m setting up:

Sysctl:

fs.inotify.max_user_watches 1000000
fs.file-max 1000000

Adding in /etc/security/limits.conf

      * soft     nproc          500000
      * hard     nproc          500000
      * soft     nofile         500000
      * hard     nofile         500000
      root soft     nproc          500000
      root hard     nproc          500000
      root soft     nofile         500000
      root hard     nofile         500000

Adding in /etc/pam.d/common-session
session required pam_limits.so

Adding in /etc/pam.d/common-session-noninteractive
session required pam_limits.so

Adding in /etc/systemd/system.conf
DefaultLimitNOFILE=500000

Adding in /etc/systemd/user.conf
DefaultLimitNOFILE=500000

What can be wrong?

8

Do I need Nginx, because I’m just using Cowboy

9

Thanx @axelson and @jakemorrison. The issue was that the parameters where to be set also in the Tsung server. I got confused because in my local setup Tsung didn’t have any problems.

2 Likes