How to get WSS connections working?

Questions / Help
1

Greetings,

I had been learning Elixir/Phoenix a couple of years ago, but had to stop. I am getting back into it again and lots of it seem familiar. Unfortunately, some of my test examples got lost somehow and I need help getting the initial connection working on HTTPS. My website is not hosted on Phoenix. Instead, the web server is hosted on Nginx and I intend to use Phoenix as an API server, connecting with sockets - No HTML will be hosted on the Phoenix side. The website is uses HTTPS only. I was able to get WS partially working (console complains about mixed HTTP/HTTPS content, but I am seeing the socket connection attempt on Phoenix, so I know something is making it through). However, I cannot get a WSS connection to work.

Here is my Nginx configuration:

upstream exchat {
  server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
}
server {
  server_name ex.mysite.com;

  listen [::]:443 ssl; # managed by Certbot
  listen 443 ssl; # managed by Certbot

  location / {
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-Cluster-Client-Ip $remote_addr;

    # The Important Websocket Bits!
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_pass https://exchat;
  }

  ssl_certificate /etc/letsencrypt/live/ex.mysite.com/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/ex.mysite.com/privkey.pem; # managed by Certbot
  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
  server_name ex.mysite.com;
  listen 80;
  return 301  https://ex.mysite.com$request_uri;
}

The subdomain “ex.mysite.com” is intended only for the Phoenix backend for API and websockets.

Endpoint.ex has something like this:

socket "/socket", ExchatWeb.UserSocket,
    websocket: true,
    longpoll: false

A socket connection attempt on my website (client-side javascript), like the following seems to show up on the Phoenix end:
const exampleSocket = new WebSocket("ws://ex.mysite.com/socket/websocket?token=undefined&vsn=2.0.0");

but not:
const exampleSocket = new WebSocket("wss://ex.mysite.com/socket/websocket?token=undefined&vsn=2.0.0");

I tried messing around with the config/dev.exs as well, but I get access denied / permission errors due to the LetsEncrypt certificate being owned by root. Not sure if any of this is even required for an API-only phoenix app with websockets:

config :exchat, ExchatWeb.Endpoint,
  force_ssl: [hsts: true],
  https: [
    port: 4000,
    cipher_suite: :strong,
    keyfile: "/etc/letsencrypt/live/ex.mysite.com/privkey.pem",
    certfile: "/etc/letsencrypt/live/ex.mysite.com/fullchain.pem"
  ],
  debug_errors: true,
  code_reloader: true,
  check_origin: ["//www.mysite.com", "//ex.mysite"],
  watchers: [
    node: [
      "node_modules/webpack/bin/webpack.js",
      "--mode",
      "development",
      "--watch-stdin",
      cd: Path.expand("../assets", __DIR__)
    ]
  ]

Do I even need LetsEncrypt on the Phoenix end for wss? Let me know what I should do here or if some of my configs should be changed to something more appropriate for my API setting.

Thanks

2

My understanding is that, if the Nginx and Phoenix servers are on the same secure private network and the Phoenix server does not accept external traffic, you shouldn’t need to run SSL on the Phoenix side. Nginx can aptly serve as a proxy that terminates SSL, reducing SSL overhead on your downstream traffic while maintaining the necessary security. Networking and infrastructure aren’t my area of focus, so my understanding could be wrong though.

2 Likes
3

This is how I have set up every server I control

4

You are right, using https option for phoenix here makes little sense if all the external traffic will come through nginx, it only creates overhead.

You need valid certificates for wss, because the certificate will be verified by the client before opening the secure websocket. You can handle all of this at nginx level without having to do any secure settings on phoenix side.

The nginx options I have used successfully with liveview are the following:

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header        X-Forwarded-For   $remote_addr;
        proxy_set_header        X-Real-IP         $remote_addr;
        proxy_set_header        Host              $host;
        proxy_set_header Access-Control-Allow-Origin *;
        proxy_read_timeout 900;

The configuration on phoenix side for deploy on my project contains this:

url: [host: "mydomain.com", path: "/", scheme: "https", port: 443],

This phoenix configuration in theory shouldn’t be mandatory for phoenix channels, because it is used for the server to understand how to do redirects behind a proxy.

5

Unfortunately, I am getting this error in Chrome browser:

test.php:12 Mixed Content: The page at 'https://www.mysite.com/test.php' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://ex.mysite.com/socket/websocket?token=undefined&vsn=2.0.0'. This request has been blocked; this endpoint must be available over WSS.

This is why the socket must be set up with WSS. Phoenix receives traffic, but it’s only going to be through the sockets. I won’t be loading HTML up through Phoenix though. Phoenix will be a sockets/API-only server. I need help getting WSS set up.

test.php will look something like:

<html>
<head>
<title>test page<title
</head>

<body>
<div>Connect to Phoenix websocket API:</div>

<script>
const exampleSocket = new WebSocket("ws://ex.mysite.com/socket/websocket?token=undefined&vsn=2.0.0");
</script>

</body>
</html>

What code block or page in Phoenix/Nginx would I set up WSS??

Also, I have LetsEncrypt all set up on the phoenix subdomain https://ex.mysite.com. If I try using that certificate anywhere in the Phoenix config, I get the permission denied error. Is there some way to make that SSL available on WSS without having to copy the file into a Phoenix directory folder every 2 months when it renews? Any trick to this?

Thanks

6

I think I got it now, I stumbled across this piece of gold: How should I configured permissions for this server - #2 by sahsanu - Help - Let's Encrypt Community Support

Looks like an automated script that copies the ex.mysite.com LetsEncrypt certificate (from the root ownership directory /etc/letsencrypt/live/ex.mysite.com into my Phoenix project (/home/myusername/exchat/priv/certs/) with the correct user ownership. I’m testing it and works good.

Important Question:

In production: Where are the SSL certificates usually stored in a Phoenix project? Do you just add them to “/appname/priv/certs” or just “/appname/certs” (if /priv is not secure enough)? What is a good secure place, and what file permissions should I use for these?

Also, I was wondering if someone could check over my Nginx config and Phoenix config and let me know if it looks right for Production, if the ports/https make sense, or anything else I should add or remove?:

Nginx:

upstream exchat {
  server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
}
server {
  server_name ex.mysite.com;

  listen [::]:443 ssl; # managed by Certbot
  listen 443 ssl; # managed by Certbot

  location / {
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-Cluster-Client-Ip $remote_addr;

    # The Important Websocket Bits!
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_pass https://exchat;
  }
  ssl_certificate /etc/letsencrypt/live/ex.mysite.com/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/ex.mysite.com/privkey.pem; # managed by Certbot
  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server { # forward all HTTP to HTTPS
  server_name ex.mysite.com;
  listen 80;
  return 301  https://ex.mysite.com$request_uri;
}

config/dev.exs:

config :exchat, ExchatWeb.Endpoint,
  https: [
    port: 4000,
    secure_renegotiate: true,
    reuse_sessions: true,
    cipher_suite: :strong,
    keyfile: "/home/myusername/exchat/priv/certs/server_key.pem",
    certfile: "/home/myusername/exchat/priv/certs/server_cert.pem"
  ],
7

Your Phoenix server should not have TLS (SSL) enabled since you are proxying traffic through Nginx. Nginx will handle the TLS termination and pass traffic as plaintext to your Phoenix server. So you should remove the certificate configuration from your Phoenix configuration and change the https: to http:.

3 Likes
8

But how do you avoid errors like the following?:

test.php:12 Mixed Content: The page at 'https://www.mysite.com/test.php' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://ex.mysite.com/socket/websocket?token=undefined&vsn=2.0.0'. This request has been blocked; this endpoint must be available over WSS.

As far as I know, the browser requires WSS, and that requires SSL set up on Phoenix?

Can you provide an example of what the config files are supposed to look like (on Nginx and Phoenix)? I tried the code examples above and they don’t seem to work for me.

9

The browser should use WSS. Here’s a little chart:

[Browser] <--- WSS over Internet ---> [Nginx] <--- WS inside the server ---> [Phoenix]

As you see, the browser will talk WSS to Nginx over the Internet. This part of the connection must be encrypted, so the browser must be instructed to use WSS and Nginx must have TLS configuration. Nginx then strips the TLS information and talks with plaintext to Phoenix, because inside the server encryption is not necessary. Thus Phoenix should be configured to use plaintext and should not have any TLS configuration.

4 Likes
10

Ok, I got it now. My sockets are successfully connecting using that chart flow.

Thanks for the help, I greatly appreciate it!

1 Like