Exploring Elixir E7: Automatic Clustering

Great episode @aseigo!

Thanks, just happy when people get something from them as well!

This is a great quick screencast thank you.

Exactly what I needed, hopefully I will find the time soon to add libcluster to my tokumei project generator. Then quickstart can be even smoother.

@aseigo, tnx for the screencasts and all the expertise you’ve shared in various communities.

• A true KDE Software and Elixir fan

Great work Arron, your screencast foo is powerful. Keep up the great work!

Amazing screencast! However I cloned the repo and can’t get it to work on development machine.
Starting libcluster on first node isn’t an issue, the heartbeat starts to appear, but when I try to run it on a second node it raises an error:

{:failed_to_start_child, Cluster.Strategy.Gossip, {{:badmatch, {:error, :eaddrinuse}}, [...] ...}

Seems there is some process already running in the required address/port and libcluster can’t start.

Hm.. which operating system, and what are the exact steps you are taking that lead to this?

Or .. do you have some other process bound to port 45892 (and probably needs to be a UDP connection ..)? You can change the default port used by libcluster by editing config/libcluster.exs and adding port: <something other number> to the exploring_elixir topology section.

Thanks for replying!

Running on:
MacOS Sierra 10.12.6
erlang/OTP 20
elixir 1.5.2
libcluster 2.2.3

Just cloned the repo and installed deps (nothing changed) and after successfully starting the first node with iex --sname node-1 -S mix and ExploringElixir.AutoCluster.start(), if i run lsof -i :45892 this shows up:

COMMAND      PID      USER   FD   TYPE   DEVICE    SIZE/OFF   NODE NAME
beam.smp     12427     me    35u  IPv4   0x1879...    0t0     UDP *:45892

Trying to start the second node iex --sname node-2 -S mix and ExploringElixir.AutoCluster.start() shows the error I previously mentioned:

** (Mix) Could not start application libcluster: Cluster.App.start(:normal, []) returned an error: shutdown: failed to start child: Cluster.Strategy.Gossip
** (EXIT) an exception was raised:
    ** (MatchError) no match of right hand side value: {:error, :eaddrinuse}
        (libcluster) lib/strategy/gossip.ex:56: Cluster.Strategy.Gossip.init/1
        (stdlib) gen_server.erl:365: :gen_server.init_it/2
        (stdlib) gen_server.erl:333: :gen_server.init_it/6
        (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

Changing the port: <some_other_number> didn’t fix the issue, I even tried to set the port to 0 so a random and available one is picked, in this way the second node can successfully start but it wont be able to automatically connect to the first or any other node.

This is my config/libcluster.exs:

config :libcluster,
  topologies: [
    exploring_elixir: [
      strategy: Cluster.Strategy.Gossip,
      config: [
        port: 45892 # tried other ports, not solving it    
      ]
      # everything else...
    ]
  ]

Maybe I’m just not getting the

if_addr: {0,0,0,0},
multicast_addr: {230,1,1,251}

configuration part, or some other setup is required to run multiple nodes in dev mode.

As far as I know a UDP socket can’t share the same address/port with other process or be restarted:

iex(1)> :gen_udp.open(8789)
{:ok, #Port<0.1174>}
# on the same or another shell ->
iex(2)> :gen_udp.open(8789)
{:error, :eaddrinuse}

but in your screencast it seems to work fine, so I’m confused haha.

UDP ports can be open by multiple processes if in broadcast mode .. but this requires binding to an interface that supports UDP broadcasting. I don’t have a Mac to test this on, but it apparently can require a bit of config .. see: Multicast overlays · gossiperl/gossiperl Wiki · GitHub

Would be interested to know if that resolves it for you …

Unfortunately that didn’t help.
According to this:

since MacOS 10.10.5, two unbound (wildcard) UDP sockets can not share the same port anymore.
So, for my project I’ll use the hardcoded epmd strategy only for development, I prefer spending more time on my app’s issues than dealing with my computer’s configuration.
Thanks a lot for your help and the amazing screencast! keep them coming!