I’m in the middle of trying to deploy a Phoenix app, and I’ve been rather surprised by how painful it is to run it without a load-balancer.
The main pain point for me at the moment is how there appears to be no facility to start the server (I use the server generated by mix release) as root to grab privileged ports and SSL certificates that only root should have access to, and then drop privileges to run as an unprivileged user.
Pretty much any other unixy server that uses certificates has this exact facility.
But here, it seems the only options open to me are:
to run BEAM as root
to use all sorts of tricks to disable the normal Linux security mechanisms protecting access to privileged ports and SSL certs
run a loadbalancer (like Traefik) in front of Phoenix to handle all the security-sensitive parts
Did I miss something? I find it hard (and scary) to imagine that people are regularly using options 1 or 2.
There is also option to use your router to redirect ports to one that you can bind as non-root. In future it probably will also became possible to use systemd socket activation to make systemd to open port for you and your will just attach to it (as non-root). Alternatively if FS allows for that then you can also set capabilities for Erlang executable.
So from an security in depth perspective is it really a huge problem to have userspace apps get access to privileged ports (option 2)? Windows does it, and I thought the major danger is if your software gets compromised it can spoof other systems on the network. But you can also lock that down with iptables rules, which you should do anyways and/or have your CSP do.
No, presuming you have a firewall, it’s just annoying and fiddly to set up, you have to dig deep into the dark lore of Linux security to get it working.
More troubling is having your SSL certs accessible to unprivileged users. It makes it trivial to extract secret keys from vulnerable servers.
Out of curiosity, wouldn’t performing encryption require the running server to have a version of the key loaded in memory? It seems like it’d just make it slightly more inconvenient to access the SSL key than a nice known FS location. Unless SSL makes some sort of “one-time sub key” from the master key initially and then uses that in a process with dropped privileges. I’m not that familiar with the depths of SSL so I’m curious.
I’m not a professional hacker, but to my understanding, compromising protected memory would be significantly more difficult than just grabbing a file.
If you follow guides or use automated tools (on Linux), they almost all explicitly set the private key files only to be accessible to root. Unless they’re all just cargo-culting, there must be a point to such measures.
You can use docker to run the Elixir app inside the container listening on unprivileged ports and have them mapped by the docker run command to the privileged equivalents.
But I agree that should be possible to start the application without the need to be root.
But mix release doesn’t start a release, only builds it:
$ mix help release
Assembles a self-contained release for the current project:
MIX_ENV=prod mix release
MIX_ENV=prod mix release NAME
Once a release is assembled, it can be packaged and deployed to a target, as
long as the target runs on the same operating system (OS) distribution and
version as the machine running the mix release command.
But I guess you know that, and you wanted to say that you were using mix phx.server to start the server.
I think you’re misunderstanding my parenthetical. I did not mean to say that I use mix release to start the server (that’s not possible). I use the server command generated by mix release. Unfortunately, I can’t edit the original post anymore to make it clearer.
Most of companies I’ve seen (regardless of programming language and framework) use a reverse proxy like haproxy, nginx or apache httpd in front of their services because of all the reasons you mentioned. You can call it best practice.
Let me turn the question around: What makes you think that Phoenix is an exception?
There is not much value for a reverse proxy for web sockets connections. If Phoenix is battle tested enough in this regard I am willing to take the plunge.
I don’t think the question was about Phoenix being battle tested but about the added complexity of having to access files as root, opening ports <1024 and dropping privileges.
All of which are also relevant for web socket connections.
There are ways to access file as root, and listen to privileged ports. The question is whether Phoenix stack is battle tested (security wise) enough to be trusted for that.
Pretty much. I never found a solution for reading the SSL certs and then dropping privileges, and I did not like the options of running BEAM as root or leaving the SSL cert files relatively unprotected either, so I ended up using Caddy as a reverse proxy instead.
You can do so using systemd (example - erlang-systemd/examples/plug at master · hauleth/erlang-systemd · GitHub). And about certs - just make it readable (and only readable) by the Phoenix process, even as non-root. If someone will achieve RCE with your service, then you lost anyway, so that isn’t something that I would worry about really.
“A full VM is thousands of times larger in lines of code and complexity than even a high performance modern proxy” do you mean the VM we run the app in or the BEAM? Wouldnt a modern proxy also run in a VM (if not on a physical computer)?
