I’ve ever heard a rumor saying that it is not recommended to expose Cowboy itself directly to the Internet. The Cowboy’s author also doesn’t recommend it, but put nginx in front of Cowboy.
It is true Cowboy has no virtual hosts nor some other features the other Web servers have, but in the point of security, is there any reason why you shouldn’t put it directly?
I think its because the permissions binding to port 80. Under most linux distributions packages for NGinX, Apache or other software are already prepared in a way to be ablt to bind that port and reverse proxy is configured in 3 or 4 lines in most of them.
Setting everything up, that your cowboy application is allowed to bind to 80 might be much harder to configure (thats an assumption, I actually never tried to).
But perhaps someone who tried can tell?
@benwilson512 has some phoenix applications in the wild AFAIK (but behind HA AFAIK)
Is there any other reasons other than port bindings?
I’m running some applications with Phoenix, and I’ve forgotten to have them had putting behind a load balancer in my environment. However, the LB just forwards the packets to the Cowboy, I can ignore about port bindings, but it is as same as exposed to the wild.
Putting it behind NginX isn’t different to a load balancer. It just forwards HTTP requests.
Well a LB does forward everything addressed to its port to cowboy while nginx is aware of HTTP and can therefore (in theory) serve statics on its own and forward only requests to dynamic resources to the cowboy, which takes some load from it. But on the other hand side, I’ve heard voices that said, the benefits of this are only theoretical but nut measurable. At least not as long as everything is on a single machine.
Also I mentioned someone who micht have more experience and knowledge about the how and why and whatnot, because he has (according to my knowledge) applications live in production.Maybe he will chime in here and give some feedback?
I’ve been happily exposing cowboy but by forwarding port 80 to 4000. I don’t see why it would be an issue other than you I wouldn’t want to run it as root.
You should never expose your application server directly to the Web. There might be unknown entry points, backdoors, possible data leaks, exploits etc.
Preferably put a filtering proxy in front and whitelist only the URIs you know are valid.
Bonus : LB + rewrite rules + caching.
Nginx is a proven solution, Hiawatha, Apache too etc.
What’s the difference between exposing cowboy directly and behind nginx forwarding all requests?
nginx as a reverseproxy will only forward requests it considers as valid HTTP requests, while a directly exposed application might get hit by garbage.
Now, the question is, whom do we trust more to not open up a backdoor when hit by garbage? Cowboy or nginx?
Also it was already said, that nginx and friends do not forward all requests, but might cache some or serve static assets on their own. Using nginx or apache, you could even mount different cowboys, rails or static sites on different pathes of the same host, without having to fiddle with those apps (except for an URL prefix in the configuration of the apps)
A bit off-topic, but, for example, HAProxy tries to “repair” invalid HTTP requests before forwarding them. It also, I think, waits for the full request to come in before forwarding it, thus eliminating some possible attack vectors.
- an HTTP normalizer : when configured to process HTTP traffic, only valid
complete requests are passed. This protects against a lot of protocol-based
attacks. Additionally, protocol deviations for which there is a tolerance
in the specification are fixed so that they don’t cause problem on the
servers (eg: multiple-line headers).
There have also been problems with ssl stack in erlang: https://blog.heroku.com/how-we-sped-up-sni-tls-handshakes-by-5x
nginx might be doing something similar.
Yay for a mention of Hiawatha. I use it as it’s easy to configure and comes with nice security-related features by default. E.g., garbage request rejection. And it uses mBed TLS rather than OpenSSL, and so wasn’t affected by Heartbleed, for example. I haven’t used it with Phoenix channels yet, but it supports websockets and hopefully there won’t be any issue there.
An article from someone with more knowledge than me: Why I use Hiawatha Webserver
That which only forward valid HTTP requests(though in the point of the view of the servers) seems to be a superior point.
This is out of mere curiosity. Compared with Nginx or some others, is Cowboy vulnerable to malicious packets or would it be any problem to crash?
In Cowboy world, each process works separately; people say those are one of Erlang’s advantages, compared with other languages. Even though the process exposed to those attacks may crash, but the other processes are not affected by it. So, I’m curious whether it would be a problem.
It’s more of an erlang thing, though. Not specific to cowboy.
In cowboy each http request gets a separate process, and then each tcp connection also gets a separate process under
:ranch, so yeah, they are pretty separate (but message passing between them might get expensive). However, crashes are not free (e.g. getting stacktraces) so it is still possible to affect the whole system.
Thanks for this thread. I’d been happily putting cowboy “out there naked” thinking that having nginx or similar in front was pointless and redundant (luckily not for anything serious yet). Now I know better and I’d never even heard of Hiawatha so I have the additional joy of exploring something new. Thank you
CRIME, Heartbleed does not crash the server process. you will have stolen information.
Do you know there is no other security hole on Cowboy or not?
I think naked Cowboy in Internet is dangerous.
I think it is the unknown security holes that are concerning, which is why the chorus here is to at least try to protect one’s server from 0-day Cowboy exploits by whitelisting URLs. I assume known security holes are patched as fast as they are able.
What’s the reason do you think it is dangerous to expose directly?
I completely agree with you. I will not expose my server either. Using a filtering proxy is a good idea. Thanks a lot!
Well, I’ve read somewhere (sorry can’t remember the source…) that one of the reasons for using something in front of cowboy is the performance for TLS. Erlang can handle it but can’t match the performance of native TLS libraries.
I agree with others here also recommending some security counter-measures like a web application firewall, garbage request denial and so on.
This got me confused and it would be of help if anyone here could answer this: if we have anything in front of cowboy, how can we setup HTTP2? I mean, it is mandatory to have a certificate, isn’t it? So, we would have 2 certificates?
Haven’t tried anything locally yet…
it is mandatory to have a certificate
Is it? https://http2.github.io/faq/#does-http2-require-encryption.
Besides, I don’t see much reason to use http/2 for anything other than static assets, and for that there are nginx, h2o, and cdns.
You may offload your public certificate on the front proxy(ies) and setup HTTP2 here.
Then use a private certificate with internal CA (freeipa for instance) between proxy(ies) and your app server(s).
That’s a possible solution, there are many others.