So I’ve been banging my head to the wall trying to figure out why my upstream was not being handled by phoenix.
I first checked my nginx configs.
Upstream set to right host and post of 4000
Proxy headers all set
Cert via lets encrypt
Phoenix app confirmed running via logs and curl on localhost:4000 on server
Http2 request coming in on 443 proxy up to phoenix
So something got me thinking, does this request fall back to http1.1 if htt2 does not work?
That lead me to look into the qa section of nginx where I found this little nugget.
Q&A
Q: Will you support HTTP/2 on the upstream side as well, or only support HTTP/2 on the client side?
A: At the moment, we only support HTTP/2 on the client side. You can’t configure HTTP/2 with proxy_pass. [Editor – In the original version of this post, this sentence was incorrectly transcribed as “You can configure HTTP/2 with proxy_pass.” We apologize for any confusion this may have caused.]
I maybe reading this wrong, but if I’m correct this is saying you can’t using http2 requests with upstreaming your request via proxy.
I than cut out my nginx stack from my app and ran cowboy as my primary http server and all is working fine now. In the end the only indiction my request even hit my server was in my nginx error logs where it simply said
upstream prematurely closed connection while reading response header from upstream
well any how I thought I would share my lesson in hopes it saves someone else a few days of their life
Application Load Balancers provide native support for HTTP/2 with HTTPS listeners. You can send up to 128 requests in parallel using one HTTP/2 connection. The load balancer converts these to individual HTTP/1.1 requests and distributes them across the healthy targets in the target group. Because HTTP/2 uses front-end connections more efficiently, you might notice fewer connections between clients and the load balancer. You can’t use the server-push feature of HTTP/2.
Those “subtle” things that may just cause a lot of debugging fun is the reason I try to avoid any moving parts that are not absolutely necessary (not using nginx with phoenix for this reason). Thanks for sharing!
I wanted to try this but cant for the life of me get basic redirection to force ssl with plug/cowboy to save my life.
Edit: looks like I figured out my roadblock, will give this a try. thanks for sharing.
Putting a webserver/reverse proxy in front of Cowboy might be necessary in order to serve more than one site/app from the same server. Caddy can automatically get SSL certificates via the ACME HTTP or DNS challenges.
Once you get used to a particular server’s/reverse proxy’s configuration it can be appealing to stick with it across different types of sites: Elixir, Python, PHP, etc. I haven’t yet looked into configuring Cowboy for TLS, but I’m interested to know how easy it is to, for example, disable TLS 1.0 and 1.1.
If you want some overload protection (like setting maximum concurrent connections) or not have Erlang handle TLS (https://istlsfastyet.com), or do a deployment without downtime, I’d recommend HAproxy, which can do pretty much anything except serve files.
So something I didn’t understand right away was that I don’t need to have encryption between the reverse proxy an elixir. Even though the upstream proxy is talking with elixir (cowboy) via http1.1 nginx is still returning h2
So in the end it still works to have nginx running in front of your app.
