Storing all genserver pid in ets table in each node

It sounds to me like you are reimplementing process groups. Using those should be easier and possibly more performant, since not every process in the group would need to create its own ETS table

will my implimentation work for 1000 nodes?

It should “work” in theory, but I don’t have 1000 nodes to test this on. Ideally, you’d benchmark this for a realistic scenario you want to handle and then decide for yourself if you need to improve performance to ensure the system operates within your operations.

You’ve made a few posts about this node structure, can you elaborate about this? This is a very large number of nodes, have you looked at Partisan? Can you discuss why you chose that architecture?

because by default, it(distributed erlang) had created a fully meshed network. so i set connect_all false and then use a key value database to assemble meshed network.

Hey @Erynn my goal was to understand why you have 1000 nodes at all, I understand that distributed erlang doesn’t work for that. Partisan however isn’t distributed erlang, so I’m curious if you evaluated that since it is designed for large erlang clusters.

1000 nodes for a chat system.

@Erynn I don’t understand how you expect to get quality answers without asking a quality question. WhatsApp ran a multi billion user chat app on < 50 servers so you’re gonna need to use multiple sentences and expand on your thinking here.

1000 nodes for a chat system. now i am trying to scale it for more nodes, so now i am trying to sync ets table to another node but for that i have to know that node pid, for that i am registering 1 genserver per node. and i am trying to keep 1 ets table per node where all nodes genserver pid is there. for that i am thinking, if 1 genserver fail means that ets table gone, then genserver registered and use Enum.each(Node.list, fn node → GenServer.cast({MODULE , node}, {:presence, {Node.self, self}}) and when each genserver module receive in handle_info and insert in that node ets table and then they use Kernel.send to send their {Node.self, self} to back. and if all nodes pid is there in each ets table then we can use another genserver to sync ets table in that node?

This is a restatement of your question, not an explanation for why a 1000 node cluster was the best way to solve your problem.

To be more direct: I think using 1000 erlang nodes is a bad idea for almost any system, and particularly when structured in a loop. I don’t think there are satisfying answers to the questions you are asking, because the questions you are asking involve a cluster architecture that is fundamentally bad.

If you can talk to us about the larger picture problems you are trying to face then perhaps we could provide some input. Otherwise I think this thread is at an impasse.

See: https://xyproblem.info/