Phoenix app in docker with ssl not receiving anything on port 443

Phoenix Forum Questions / Help
, , ,
1

Hey there fellow devs,

For the love of everything, I can’t seem to get this to work… I have a docker container with a phoenix app running in there. It’s hosted on a subdomain “sub.example.com” and I need to serve it with https.

So I got myself a wildcard SSL certificate, installed it, configured the production file and exposed 80 and 443 ports of the docker container.

Tried it out and port 80 works fine, but 443 is always returning a “ERR_CONNECTION_RESET”. The logs is showing nothing on 443, but 80 works fine.

Been trying for awhile now, and now i need your help. Any idea on whats wrong? Check the code below:

Dockerfile

FROM bitwalker/alpine-elixir-phoenix:latest

# create app folder
RUN mkdir /app
WORKDIR /app
COPY . .

# setting the port and the environment (prod = PRODUCTION!)
EXPOSE 80
EXPOSE 443

# install dependencies (production only)
RUN mix local.rebar --force
RUN mix deps.get --only prod
RUN mix compile

prod.exs

config :example, ExampleWeb.Endpoint,
  http: [port: 80],
  url: [host: "sub.example.com"],
  cache_static_manifest: "priv/static/cache_manifest.json",
  https: [
    cipher_suite: :strong,
    otp_app: :example,
    port: 443,
    keyfile: System.get_env("SSL_KEY_PATH"),
    certfile: System.get_env("SSL_CERT_PATH"),
    cacertfile: System.get_env("SSL_CHAINED_CERT_PATH")
  ]

curl https://sub.example.com/ --verbose result:

*   Trying 64.225.24.82...
* TCP_NODELAY set
* Connected to sub.example.com (64.225.24.82) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to sub.example.com:443
* Closing connection 0
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to sub.example.com:443
2

How are you running the container? EXPOSE doesn’t do anything, really:

EXPOSE

EXPOSE <port> [<port>/<protocol>...]

The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. You can specify whether the port listens on TCP or UDP, and the default is TCP if the protocol is not specified.

The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports.

1 Like
3

This suggests that the TCP connection to the Phoenix app was established, but it was closed before any TLS handshake messages were sent by the server. This usually means there is a problem with your certificate/key files: Erlang’s :ssl does not verify the locations at startup, but only once the files are actually needed during the handshake. If it can’t read the files, the ssl socket crashes, and curl reports what you see.

So check if the env vars are correct, the files are present and readable and the private key is not encrypted (password protected).

4

Something else to think about is to let Nginx or other web server handle the SSL part, and proxy_pass requests to upstream. Then you can just run your Phoenix app on port 4000 and not worry about configuring ssl in Phoenix.

This is how I run my phoenix site with nginx in between.

upstream myapp {
  server localhost:4000;
}

server {
  server_name myapp.de;
  listen 443 ssl http2;
  listen [::]:443 ssl http2 ipv6only=on;

  ssl_certificate /etc/letsencrypt/live/myapp.de/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/myapp.de/privkey.pem; # managed by Certbot
  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

  client_max_body_size 50m;

  location ~* \.(eot|ttf|woff)$ {
    add_header Access-Control-Allow-Origin *;
  }

  location ~* ^.+\.(css|js|ico|jpg|jpeg|png|svg|woff|woff2)$ {
    root /var/www/apps/myapp.de/current/lib/myapp-0.1.0/priv/static;
    etag off;
    expires max;
    add_header Access-Control-Allow-Origin *;
    add_header Cache-Control public;
  }

  location ~ live/websocket {
    proxy_http_version 1.1;
    proxy_set_header Origin '';
    proxy_set_header X-Forwarded-Host $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_pass http://myapp;
    break;
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://myapp;
    break;
  }
}
2 Likes
5

I agree with @egze.

6

That’s interesting to know! I genuinely thought it made a different, lack of reading on my part. But I am running the docker with:

docker run -it -p 80:80 -p 443:443 --entrypoint=bash --name example-app example:1.5

@voltone The ENV variables are being read. I’ve test the paths by making a typo and it crashes immediately. I am not sure how else I can try it out, but I am not getting any type of errors for guidance.

@egze This is interesting, iv’e been searching the web and seen this pop up several times. Thinking about resorting to this method. But then I ask myself, why even have the option to add SSL via Phoenix?

8

Because you can also use pure Erlang/Elixir to host your site if you want. SSL can certainly be done with pure Phoenix. Just that for me it’s not so practical. I host multiple sites on one box and I only have one 443 port :slight_smile:

1 Like
9

So have installed nginx and configured it to match the phoneix needs as much as I could, but now I keep getting a 400 bad request - Request Header Or Cookie Too Large. I went through the internet and tried all kinds of settings and I just can’t get it to work. (Excuse my skills, I am still new to devops related things)

My current configs are as follow (Without ssl for now, trying to make it work the normal way first to try and understand it):

etc/nginx/conf.d/default.conf

upstream phoenix {
  server localhost:4000;
}

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}

server {
  listen 80 default_server;
  listen [::]:80 default_server;
  server_name localhost;

  large_client_header_buffers 4 16k;

  location ~ live/websocket {
    proxy_http_version 1.1;
    proxy_set_header Origin '';
    proxy_set_header X-Forwarded-Host $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_pass http://localhost;
    break;
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://localhost;
    break;
  }
}

config.exs

config :myapp, myappWeb.Endpoint,
  url: [port: 4000]
10

Hard to say. Does it happen when curl-ing? Maybe you have some huge cookies saved in the browser.

11

Currently using chrome in incognito and doing a hard refresh

12

But try curl. Then you are sure

13

Tried with curl curl http://localhost/ --verbose
and I got this:

*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Server: nginx
< Date: Fri, 08 May 2020 15:04:55 GMT
< Content-Type: text/html
< Content-Length: 226
< Connection: keep-alive
<
<html>
<head><title>400 Request Header Or Cookie Too Large</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>Request Header Or Cookie Too Large</center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host localhost left intact
* Closing connection 0

I have also tried with with the docker port with 4000, and that works fine! Might be because I can’t use port 80 on my local pc?

14

@egze Found the problem! Got it working for port 80 now, next step is 443. Not sure why it gives the cookie error, but I had the wrong configuration. The one that works is:

upstream phoenix {
  server localhost:5000;
}

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}

server {
  listen 80 default_server;
  listen [::]:80 default_server;
  server_name localhost;

  large_client_header_buffers 4 16k;

  location ~ live/websocket {
    proxy_http_version 1.1;
    proxy_set_header Origin '';
    proxy_set_header X-Forwarded-Host $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_pass http://localhost:5000;
    break;
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://localhost:5000;
    break;
  }
}
15

Got a bit further but now I am getting a “502 Bad gateway” error. Current configs are:

Phoenix production config:

config :example, ExampleWeb.Endpoint,
  http: [port: 5000],
  url: [host: "example.com", port: 5000],
  https: [
    otp_app: :example,
    port: 5001,
    keyfile: System.get_env("SSL_KEY_PATH"),
    certfile: System.get_env("SSL_CERT_PATH"),
    cacertfile: System.get_env("SSL_CHAINED_CERT_PATH")
  ]

NGINX config

upstream phoenix {
  server localhost:5001;
}

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}

server {
  server_name localhost;
  listen 443 ssl http2;
  listen [::]:443 ssl http2;
  

  ssl_certificate /etc/nginx/ssl/example_chained.pem;
  ssl_certificate_key /etc/nginx/ssl/example.pem;

  large_client_header_buffers 4 16k;

  location ~ live/websocket {
    proxy_http_version 1.1;
    proxy_set_header Origin '';
    proxy_set_header X-Forwarded-Host $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_pass https://localhost:5001;
    break;
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass https://localhost:5001;
    break;
  }
}
16

You don’t need any https for phoenix at all. It runs only with http. https is for nginx, but the internal traffic between nginx and phoenix doesn’t need to be encrypted anymore. You can certainly encrypt it if you want, but what’s the point?

To summarize:

internet   -----[https]----->   nginx   -----[http]----->   phoenix
17

Trust. Its always about trust…

Consider you are in a cloud environment where LB/RP are not on the same host as your application. Its quite common to use HTTPS in the backend then:

  1. Ensure we are talking with a trusted backend server
  2. Ensure that user data isn’t sniffable by the cloud provider or anyone else who might have access to one of the network layers (virtual, physical, over-/underlay, whatever).
1 Like
18

Good point. I assumed it was the same physical server.

1 Like
19

Alright, after days of searching, trial and error and discussions on this post. It finally works with the following configurations! Thanks to everyone and especially @egze with helping out and sparring!

Btw both nginx and phoenix are in the same docker conatiner.

Phoenix - prod.exs

config :example, ExampleWeb.Endpoint,
  http: [port: 5000],
  url: [host: "sub.example.com", port: 5000]

nginx config

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}

server {
  server_name sub.example.com;
  listen 443 ssl http2;
  listen [::]:443 ssl http2;

  ssl_certificate /etc/nginx/ssl/example_chained.pem;
  ssl_certificate_key /etc/nginx/ssl/example.pem;

  large_client_header_buffers 4 16k;

  location ~ live/websocket {
    proxy_http_version 1.1;
    proxy_set_header Origin '';
    proxy_set_header X-Forwarded-Host $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_ssl_server_name on;
    proxy_pass http://0.0.0.0:5000;
    break;
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_ssl_server_name on;
    proxy_pass http://0.0.0.0:5000;
    break;
  }
}
4 Likes