Hello, community!
I’ve been using libcluster to connect nodes to the Elixir cluster via Kubernetes deployment, My strategy to detect the inner/outer nodes while scaling up /down is Cluster.Strategy.Kubernetes.DNS.
while I make scale up/down, and my app has some logic based on that, I need to broadcast some data to all available nodes on my cluster(I’m using an ERPC call to do that).
However, I’m facing an issue in the following case, When I scaled down the deployment, I supposed the libclutser should disconnect the node when the k8s pod starts to terminate, but after the node was disconnected, it went back again to the cluster! and be visible and I can see it by Node.list() until the be pod is fully terminated!
My problem is while the node doing the graceful shutdown, I need to prevent any ERPC call on it, and I’m trying to figure out a way to prepare the node for shutdown, as I mentioned above (the node is visible on the cluster), it causes me the following error(“raised an exception ** (ErlangError) Erlang error: {:erpc, :noconnection}”)
Also, I know while I’m reading about the Erlang and beam VM,
“BEAM nodes connect to each other when a message is sent between them. Due to this, disconnected nodes automatically try to reconnect as long as messages are still pending for the disconnected node”, So based on that I supposed what makes the node connected after the libcluster disconnected may be an ERPC call was pending.
At the end is there a way to make sure about disconnecting the node from the cluster and prevent any ERPC call on the node while it is in termination mode?
Thanks in advanced.
New Update:
I’ll try to overwrite the beam vm args and set -kernel dist_auto_connect never, and see if I can achieve my goal by that, but is there any other impact?
