How to use an Elixir release without epmd

The title says it all:

I’d like to completely deactivate Erlang port mapper daemon for my application. Since it is running in a shared hosting setup, as far as I know, anyone could connect to it without any kind of authentication.

Also this particular application will never run in a clustered setup, so I think it is safe to say, that empd is not needed in this scenario.

Mix release supports the environment variable RELEASE_DISTRIBUTION=none so using that when generating the release should disable distribution features. I don’t know if it avoids starting epmd but at least you shouldn’t be able to connect to the release.

I’m not a 100 % sure on these details so test before going into prod. Hoping someone wiser can chime in too.

Source: https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-environment-variables

3 Likes

@Nicd just to be clear: This also means that it is entirely impossible to connect to the running system, right? So no introspection of the running system, even when connected directly to the machine the node runs on?

I think so, because AFAIK that connecting like remsh uses the distribution system. But you should try it and report. :slight_smile:

As soon as I have time on my hands again :slight_smile: thanks for clarifying

https://www.erlang-solutions.com/blog/erlang-and-elixir-distribution-without-epmd.html

The article is a bit old but perhaps still valid?

1 Like

Look into blocking the EPMD port at the interface (with iptables or equivalent new hotness); that way you can still SSH into the box and attach an IEx session like usual.

1 Like

First, you can have epmd and the node itself bind to 127.0.0.1 so it is inaccessible outside of the host.

Next, there is https://github.com/tsloughter/epmdless which can be used to simplify running a node that you can get a remote connection to still locally but does not have epmd running.

Finally, OTP 23.1 will make this even simpler, removing the need for epmdless entirely in the case that you are fine with a static port being used for the Erlang node.

3 Likes

This looks great. Thanks for sharing!

And also thanks to all the others for chiming in. I’m planning on trying all solutions suggested here and to report back when that happened. Thanks for all the contributions!

1 Like

Ah @al2o3cr think this is a misunderstanding: The release is running on a host where multiple users can connect to via ssh. iptables would not help me in the scenario, if I’m not mistaken, since everyone who can ssh into that server would be able to circumvent the firewall.

  1. RELEASE_MODE=none still starts epmd, unfortunately.
  2. As stated before, iptables won’t help due to the shared hosting environment.
  3. While epmdless is indeed working as advertised it has the same problem as 2. Other users could still connect to the running application just by inspecting the started process and reading the cookie from there (at least that’s my understanding)

@tristan is my understanding of epmdless correct here, or am I still missing something?

Thanks!


If a moderator strolls by: could you please fix the topic of this post to contain epmd instead of emdp? :man_facepalming:

The env var is called RELEASE_DISTRIBUTION, did you try that?

Oh man… my typing game is currently an atrocity. Sorry :smiley:

I ran MIX_ENV=prod RELEASE_DISTRIBUTION=none mix release and _build/prod/rel/epmd_deactivation/bin/epmd_deactivation start which shows epmd when running netstat -tulpn | grep LISTEN.

(I also used killall epmd and double checked with netstat to make sure that running this release was triggering epmd - which it did)

1 Like

Yes, if the machine’s root or the user under which the node runs is compromised then a user can easily connect to it. You can technically get it to not be in the ps results but it’ll still be in a file or env variable which would be just as compromised.

1 Like

Hm. If only the root user can do that it wouldn’t worry me so much. If that user is ever compromised you’ve lost anyways…

I thought this would also be a problem with regular users, since they can inspect running processes (I tried it with another user on my machine, who could simply see the complete connection string in htop).

If I could put the connection secrets into a file that would indeed solve the problem I think. Because I can then restrict read access to the user who is running the software, which would be good enough.

So how can I do that? :smiley:

Not sure with a mix release.

It is a feature I’ve been meaning to add to relx’s start script since someone brought up the same concern in the issues on it a little while ago.

The basic release run script isn’t too complicated, if it is of serious concern at the moment you can write your own – not sure how mix lets you include those in a release, but I’d assume it does somehow. Thats what I suggest to rebar3 users at the moment.

I guess so many of us are running in containers these days we see it as only being able to see the process if you can gain access to the containers pid namespace :slight_smile:

But if this isn’t possible with mix’s release script I’d suggest opening an issue. The script can be changed to rely on a .erlang.cookie file I think is the easiest way. But the key is to not read it in in the script if the user doesn’t want :). rebar3 has that issue, we support the file but we read it in to a variable and pass it to -setcookie. Not as simple a fix for us since we need that variable for other op actions at this time but Elixir shouldn’t since Jose put all that directly in iex.

1 Like

Success! (partially) :wink:

I used https://github.com/oltarasenko/epmdless-elixir-example as a starting point and removed all occurrences of “cookie” from the vm.args and the config.exs files.

This leads to the erlang vm falling back to the cookie file $HOME/.erlang.cookie which must be chmod 600 in order for the application to start.

When I tried to connect from an “attacking” node, the connection was refused with a [error] ** Connection attempt from disallowed node :"d@127.0.0.1" **

That is great, because then, as far as I understand it, the application should be as safe as you can keep this cookie file. I think this is as safe as it can get in the setting I’m dealing with.

I’m still not sure though if this will work with mix release or what I’ll have to do for that.

1 Like

Ah nice. Did you use distillery then? If so, should try patching the start script created by mix release and then PR’ing that, or talking to Jose about it :slight_smile: