Running local observer against a remote Docker

Questions / Help
, ,
1

Having a lot of trouble getting this to work with a local Docker instance… I am trying to basically do the following:

  1. Run docker-compose up (on my machine or a remote)
  2. Start a local iex and start :observer
  3. Use the observer gui to connect to the remote host

I have figured out I need to do the following (but theres likely more I’m missing)

  • Add :runtime_tools to the :extra_applications section of my mix.exs file.
  • Expose the epmd port 4369 (maybe? I read somewhere it doesn’t allow external connections)
  • Set the cookie and name of the app

I cannot get a Node.ping/1 from my local to the docker instance to go through. Connecting from within the docker instance works fine but it doesnt have the display necessary to run observer (and I don’t want to forward it because I plan to use this against a remote docker in prod and I dont want the observer gui included). What am I missing?

Here are some of the resources I’ve come across (none of which have been any help)

1 Like
2

My recommendation would be to use ssh and forward the ports across that connection, it won’t require you to expose the node, the connection will be encrypted, and you can secure it much better. To connect remotely though, you need to expose the epmd port, as well as the ephemeral range used for individual sessions, and make sure to configure that range in the kernel settings for distribution (you can find those in the erlang man page for the kernel app). I believe that’s all that’s needed. You can definitely connect to epmd remotely, that’s key to how erlang distribution works.

3

Yeah so I changed the epmd port in my docker instance so it wouldn’t conflict with the one running on my local machine by setting the environment variable ERL_EPMD_PORT to 43699 instead of the default 4369. But it sounds like I need to expose the port in which erlang distribution connects through. When I connect to the docker instance and run epmd -names it outputs a random port value, how can I set that to a specific range that I can then expose through docker?

1 Like
4

Ok so I found how to set the min and max ports, heres my vm.args file:

-sname myapp
-kernel inet_dist_listen_min 6000 inet_dist_listen_max 6999
-cookie myapp

And my Dockerfile has

EXPOSE 6000-6100

With my docker-compose.yml

    ports:
      - "4000:4000"
      - "43699:43699"
      - "6000-6100:6000-6100"

And when I connect to my docker instance and run epmd -names I see the following:

$ docker exec -it f231ddad1d52 /bin/bash                                                                 ‹ruby-2.3.3›
root@f231ddad1d52:/app# epmd -names
epmd: up and running on port 43699 with data:
name myapp at port 6000
root@f231ddad1d52:/app#

But when I connect from my local:

$ ERL_EPMD_PORT=43699 iex --sname myapp --cookie myapp --hidden                                        ‹ruby-2.3.3›
Protocol 'inet_tcp': register/listen error: epmd_close

:cry:

5

The reason it doesn’t work is because epmd will only listen on the loopback interface. When connecting from your machine, docker binds the exposed port over the bridge interface, and epmd rejects it because of that.

So far I haven’t found a working solution, but I’m on it. Will report back if I get it running.

1 Like
6

Found anything on this? I’m not sure I understand how EPMD would work if it only ever listened on loopback, how would machines connect then?

7

I have them communicate via SSH Tunnels, but you can always set ‘public’ names -name too. I highly recommend having EPMD communicate over tunnels as even just being publicly accessible can crash the BEAM systems via a variety of ways even before they connect a node.

8

This has been a lifesaver

https://github.com/chazsconi/connect-remote-elixir-docker

3 Likes