I find myself quite often in the following situation.
Colleague: “Elixir/Erlang is problematic because we cannot secure the VM (i.e. Beam). It is vulnerable to code injection because anybody that gets to the system can start a remote shell and change running code without anybody noticing it. On the JVM in contrast this is not possible.”
To some extend this argument is correct. The JVM (and most other runtime systems) don’t offer the debug or cluster capabilities BEAM offers which make debugging a lot harder but also works in favor of securing the system (completely neglecting the code that is actually running withing the VM).
My question is: What are good arguments in favor of the security of BEAM and what are best practices to secure a running system (both in cluster mode and as a single isolated VM) as much as possible?
To be honest… But once the attacker has access to the system they can do anything… I do not need to alter the JVM to read its memory. To sniff network traffic, to get hands on certificates, to well, achieve about everything…
An attacker that came thus far, does not care anymore about the runtime. They care about the tasty stuff. SSH keys, config data (which usually contains passwords for DB or other external services), business data…
While what you say is true, @arnomi is not talking about the system running the VM, instead he is concerned about the VM itself, aka about what you can do when you get access to it without going through the server running the Erlang VM, eg: maybe like guessing the Erlang Cookie or breaking into it from the code(if this is possible!!!)
Its both. I think I am mostly asking for: What are good arguments against the claim “runnig Beam is not secure”, (which by itself is not a well formulated claim, but one I have heard quite a lot).
And while on the topic, I think its interesting to understand best practices for securing a BEAM based system.
For example, one of the arguments against “you have lost if someone is in your system is: you at least might have monitoring on file level changes and intrusion detection systems. So if you are lucky you might catch this in time.” On the BEAM you cannot easily monitor (if at all) changes applied to the runtime system.
If you remove those debugging capabilities, by for instance not running distributed Erlang and not using run_erl/to_erl, you get into a very similar state as the JVM. You cannot debug, but you also cannot do code injection.
Is there anything you can do when running in cluster mode? My guess is the best protection would be to enable SSL for the distribution in which case you would at least need access to a proper certificate for connecting.
Is there anything that can be done from within the VM? For example monitoring code changes and sending notifications if a module gets reloaded?
Distribution over vpn is another alternative to tls (which I find simpler since no certificate management), wireguard will probably be integrated into the linux kernel “soon” (maybe in 5.0?). There’s also tinc, but it’s slower. I can publish a demo repo with my setup for both, if there is any interest. And if your hosting provider also provides a private network between vms as a service, then it’s even easier.
to the config to avoid epmd binding to all network interfaces. I also close all ports by default, and only leave the ones necessary for the application usage open.
Do you have a good blog/video guide on how to tighten / selectively disable security-sensitive BEAM features? I for one I am not at all interested in Distributed Erlang because I am mostly doing small-to-medium apps and I’d like to have the distributed stack fully disabled until it’s very clear that I need it.
And yes, removing debugging (even though that’s an awful idea).
No, not really. If you don’t run distribution and turn off the shell, then the only other entry points into the system are what you create in your application.
I realise this isn’t your job but I am just gonna bounce the idea with you: Erlang’s BEAM configuration can really use a few cheatsheet-like resources. We can all go and gather the necessary info on demand of course – but I for one love it when such a beautiful and complex piece of machinery (like the BEAM) also comes with a few config shortcuts for commonly requested use-cases.
What are the commonly requested use cases? Running the VM with default settings is the common case and everything should be well-tuned towards that.
It may be that we are venturing into “unknown unknown” territory, which is why there is a disconnect, but it is worth saying the erl command line reference is quite complete: Erlang -- erl
EDIT: I removed my reply to @arnomi as I misunderstood the question and @garazdawi answered it correctly next.
I would put a meta trace on erlang:load_module/2, and then another meta trace on erlang:trace/3 to know if anyone disabled the tracing There are probably still a few ways that you could load modules without noticing it, but that should cover the obvious cases.
In this case, on top of what you said, I’d add: version without distributed features, security-hardened version, debug-less version.
IMO such use cases are common enough. Do you disagree?
What I was saying is that a singular cheatsheet with bullet points like “start the BEAM like so if you don’t want distributed features:…” would be very useful.
And I already acknowledged Erlang’s excellent reference documentation – never criticized that. I only said I personally would be overjoyed if there were a few shortcuts through it.
Right, that’s my point. The distribution and debug features are opt-in. When you start the VM, they are not enabled by default. You don’t have to do anything.
