you can’t expect high skill and experience from 17 years old boy.
Erynn
Erynn
see i am working for someone so i can’t say for whom i am working.
but i will explain what i am trying to do, see this application have group which contain something like 2 lakh people and and we are thinking to increase it. we are using mnesia to keep user session pid with user detail in mnesia to broadcast to all online users. but if we increase group size, then we know mnesia will not work, write and sync in mnesia is wrost when size grow so we are thinking of using ets table because it is fast.
the main problem is sync
0
dimitarvp
sbuttgereit
Tyson
I’m going to sidestep the political discussion, assume that 1000 nodes is a good approach, ignore that the ring connection strategy sounds questionable, and just address the question:
It seems to me you don’t need to maintain a PID lookup table on every node, you just need one central place that knows all the PIDs. This is like a “service discovery” strategy. So each node registers itself (or the PID of its ets-managing GenServer) at startup with this central registry, which could be an additional node, a standalone service/database, or whatever. It wouldn’t require anything exotic to handle requests from 1000 “clients” (the worker nodes), especially if requests are only fired at node startup.
Though it’s not clear to me whether you need to look up PIDs only to recover a failed node or if you have to look them up every time you want to sync any changed local data. If it’s the latter, you might overwhelm the service registry with lookups.
1
lud
Well that node would have to be connected to the 999 others, not sure if itis prnctical.
Now @Erynn what I do not understand is why you want to keep all the PIDs on all nodes if they are not connected together. Do you connect and disconnect if they need to talk?
Plus if you have 1 unique genserver per node that should always be here, you can register it with a local name, so you do not need the pid, you always know that on each node there is that process.
Maybe you should describe how you want to work with the node ring and how data transits along the ring.
0
Tyson
Was just coming back to mention this 
Notice @Erynn that a GenServer started with start_link(..., name: MyLocalAtomName) is addressable with GenServer.call({MyLocalAtomName, :"my_node_999@my_host"}, ...) as long as the node names are known ahead of time.
You’re right, it shouldn’t be part of the BEAM cluster at all, but a standalone service, e.g., HTTP REST API.
1
lud
You’re right, it shouldn’t be part of the BEAM cluster at all, but a standalone service, e.g., HTTP REST API.
Ah yes.
In that case a tool like zookeeper is already made for that exact use case I believe.
0
Erynn
yay by using etcd
that’s the thing i am talking about but GenServer.call will increase overhead so to reduce overhead we can use GenServer.cast and and receiving end they use Kernel.send to send data back.
0
Tyson
Ah, I see you did figure that out already. But then why bother updating PIDs on other nodes? You don’t need them if you can send messages ^this way.
If it were me, I would want to gather some benchmarking/profiling data before assuming I can outperform GenServer.call()
1