Why is Registry local only?

Just use gproc. If you don’t like the API, wrap the parts you don’t like in a better abstraction

Admittedly, my only experience with gproc is using this part:

https://github.com/uwiger/gproc/blob/master/doc/gproc.md#reg3

Ultimately, gproc is the answer. It has already solved the problem (and you will end up running into problems it has already solved). Take the functions you already created and use them as wrappers around the gproc calls.

The simple (and boring) answer is that distributed registry is really, really hard to get right. The OTP team is already working on improving what’s available in Erlang itself, so there’s no need to duplicate efforts.

I realize that gproc has richer features but I still don’t see the showstopper for using :global.register_name/2 (with :global.whereis_name/1 or {:global,term}) in the meantime - obviously I’m missing something.

Are you just using Registry like a pubsub system?

If you’re using it for a PubSub then you are probably running a :duplicate registry, in which case there are better ways in handling Broadcasting across nodes than using Registry.dispatch.

If you want to build it yourself and still use most of your Registry implementation then what you need to do is have a Broadcaster at each node that is part of a :pg2 group or something. Then this Broadcaster is actually in charge of dispatching messages.

When it receives a local message it broadcasts the message to the other Broadcaster processes in the :pg2 group then runs Registry.dispatch/3 locally. When it receives a message from another node it just runs Registry.dispatch/3 locally.

That’s basically what phoenix_pubsub does. If you need something more than just PubSub that Registry provided then I would still recommend a per node Orchestrator process of some kind.

Once again though, this is for :duplicate registries. If you are using a :unique registry then @michalmuskala comment about registration being hard becomes the plain fact of the matter.

I am using :duplicate, and I am using it for pub sub, but not every process gets the same data when an event fires. When they subscribe they send along an mfa, and then when an event occurs, the process takes its potentially large state and runs the mfa on it to summarize it into a smaller summary that goes to the subscriber. This is because the summary is different for each subscriber due to permissions or timezone of that particular user. Like if a user needs to get a count of calls that were presented to him since midnight in his timezone. Versus another user who needs any call at his company since midnight in some other timezone.

That means that I can’t just use a dumb Broadcaster because I would have to send the entire state of each process to every node before it could be summarized, with no guarantee that any process even subscribed to that data.

As for gproc, I’ve messed with it several times and I cannot get it to work. It seems to have what I need but when I try to use any global function I get the following error.

** (ErlangError) erlang error: :local_only

And I just don’t know what that means. If I could get past that I think I could maybe make this work.

This maybe?
[erlang-questions] How do I start gproc in a global way?

I think maybe you’re using an :l atom where a :g atom should be used instead.

But the way your constructing your system seems interesting. I probably would have done it the other way around, and have the map/reduce offloaded onto the subscribing processes.

Either way, if you can give me an example of the call your making that throws the error I may be able to tell you what’s going on.

A very quick rundown of your options:

• Registry is local only
• gproc has a global mode but the consensus is that it is unreliable. so consider it local only.
• you can use pg (part of OTP) if you need a distributed process group (duplicate keys)
• you can use global (part of OTP) if you need a distributed process registry (unique keys)
• you can also use syn - you seem to already be aware of its pros/cons
• you can also use Phoenix.Tracker, which is part of the phoenix_pubsub project, as a distributed process group (duplicate keys)

EDIT: Updated in 2023 to mention pg instead of pg2 in Erlang/OTP.

This is probably why I was having a problem with gproc. I guess I’ll fool with it and see if I can’t get it working. If gproc defaults to local only mode that would certainly explain why it wouldn’t work.

For those that have used gproc, what happens when you need to add an additional node? Would it just know about that node the first time a process on the node tried to use it?