Is Horde Registry really distributed?

I have never heard of horde before but it sounds interesting.

Perhaps you need to use horde/cluster.ex set_members to create a cluster?

Make your own assessement if you believe it to be production ready.

It is unclear what problem you’re trying to solve but you should not look past the robust Erlang pg registry. There is also a distributed supervisor based on Erlang pg called pogo. It is likely to be a more robust foundation to base a solution on.

For establishing your cluster lib_cluster is excellent and offers a range of discovery methods.

If you have a large geographically dispersed cluster and need greater control over distribution there is also partisan.

Nothing stopping you from storing a remote pid in an ets tables. You just can’t directly access contents across the cluster. You can deposit the same pid in multiple ets tables under the same key to ensure availability.

A few things to remember:

• IIRC pg uses two phase commit. It might be slow and I’m not entirely sure the soundness has been proven.

• horde and swarm use CRDTs to back state. So note that your cluster can be inconsistent if there is communication lag. The consistency paradigm is eventual

Tread carefully!

If I had a wishlist, a distributed supervisor based on VSR would be it (raft has serious problems)

You may be referring to the old slow pg that pg2 replaced.

The docs for the ‘new’ pg in OTP/23 that deprecated pg2 explicitly states that pg is eventually consistent. The source code for pg indicates it was contributed by WhatsApp.

pg doesn’t have a long list of critical crashing problems unlike some other options mentioned here.

In theory, theory and practice are the same. In practice they are not.

I have been working on a comparable library ProcessHub. Although it’s still in the alpha stage, active development is ongoing.
I’m not going to mention the other libraries already pointed out but alternatively, you may want to explore Swarm.

While you could develop your implementation using the pg module, which is robust and would perform well in your described scenario, you may encounter challenges when the distribution model becomes more complex.

thank you for engaging @cmkarlsson

not really, Horde has a NodeListener genserver implementation for auto handle incoming X outgoing nodes.

And as @adw632 mentioned, we have a mature implementation for strategies for cluster discovery/formation
https://github.com/bitwalker/libcluster

Thank you @anuaralfetahe , but Swarm seems to be with a huge list of issues… and low maintenance, just as horde.

As for ProcessHub , congratulations for your effort, I’ll sure give it a try.

But I need some more mature and tested. And pogo seems to have a use case with whatsapp.

As for partisan . I really think the project is far too much complex for my needs. I really prefer to go with default distributed erlang for now.

I think maybe you’re confusing the difference between cluster formation and distributed registries.

Horde is a distributed registry and relies on a clustering implementation to actually get nodes talking to each other, hence the mention of set_members. I’m guessing you need a cluster configured. I’ve run Horde in production and if your cluster is configured and the registries are listening then even if you somehow were to double register on separate nodes for the same name before it was synced, one would get killed once it did sync. If you’re not seeing one process get killed for a name conflict then your horde registries are not getting node cluster updates.

Thank you for engaging @felix-starman

As I mentioned before, I really do have a cluster formed in a k8s cluster using libcluster.
I really can see process distribution of horde working very well. But the registry does not.
I spawn in my 3 nodes the same process with the same key in my 3 nodes, and Horde.Registry gives me a different pid for each node. AND does not know the other nodes keys.

Hello, I’m the author and sole maintainer of Horde.

Just popping in to answer a few of your questions:

Is Horde Registry really distributed?

Yes, it is backed by DeltaCRDT, which has leaderless syncing. Updates that come in are written to an ets table for quick read access. Writes go through the CRDT.

Am I missing something?

It looks like you have configured Horde.NodeListener for your Horde.DynamicSupervisor, but not for your Horde.Registry. I checked, and the guides are not super clear on this, probably owing to the order in which things were built (Horde.NodeListener was added after the guides were written).

Others have mentioned this already, perhaps you should re-read their responses.

Unmaintained

Horde is a library that, as I have learned, has some limitations, but still remains useful to quite a few people (>9k downloads last 7 days). I don’t have the time nor the interest in large development efforts for the project, and there are not a lot of new PRs coming in to the project.

I try to stay on top of things, but the obvious bugs have been ironed out a long time ago, and aside from a few small issues, I would say that Horde is doing fine.

Finally, I will note that Horde is provided free of charge and without any kind of guarantee that it will work.