Swapping out dist epmd et al - making a non point-to-point cluster

Greetings! I actually asked a similar question a long time ago but I couldn’t find it in search.

Basically what I want to do is make it so that two Elixir nodes can communicate with each other using a central broker rather than establishing point-to-point connections. I’m fine writing all the broker client stuff.

I assume that I’ll have to create my own registry, which should be easy. The next thing I need to do is replace the plumbing used by send. I vaguely remember someone here saying this is possible but again I couldn’t locate the post.

To sum up - is it possible to replace the dist protocol implementation with my own (that still adheres to the code contract)? If so, how do I do such a thing or is there a reference I can go to for this?

I think I might be able to get what I want from a libcluster strategy, but it still looks like ultimately process sends would need direct connections. If it doesn’t make sense to change the clustering requirement of point-to-point, I’d love to know that as well.

For context, I am in a situation where my Elixir nodes will be in environments where they can only make outbound connections and I still want to be able to auto-form a cluster. I thought by making this outbound connection to a message broker, I could do just that.

Not the answer you are looking for, but have you thought about using something like Tailscale to create an overlay network where all nodes are outbound (but the brokers)?

That’s actually a great idea. I’m a huge tailscale fan.

You can take a look at epmdless for a way to run without EPMD, and implement a custom distribution protocol to replace the standard one.

2 Likes

IIRC that behaviour can be now used without any external libraries as it is part of the main distribution