jaybe78
Ditributed cluster: up to how many nodes we can scale using a library like libcluster?
Hello,
I would like to know up to how many nodes max we can run a distributed app with a simple libray like libcluster ?
I read somewhere that after 40/50 nodes, you can start having issues like loss of messages and it generally does not scale well above that number.
For the context:
I’m currently building an app with an expectation of 2/3 millions concurrent users.
Those users would spread around 300 000 + channels.
So each channel would contain something like 100 users or less joining and they would mostly exchange chat messages.
Phoenix Presence is set up on those channels so an user can be aware at anytime of whose connected.
I heard about an Erlang library called Partisan that is used for very large cluster, but I guess for my use cases lib cluster should be enough ?
If anyone has been working in a distributed app it would be nice to share your feedback here.
Cheers.
Most Liked
mpope
Fair enough, thought for food.
For automated discovery specifically take a look at GitHub - nerves-networking/mdns_lite: A simple, no frills mDNS implementation in Elixir · GitHub
jaybe78
yes thanks definitely interesting ![]()
pleb
Virtual nodes are spread across physical nodes, and the dynamic partitioning scheme allows physical nodes to be added and removed while virtual nodes always find the closest physical server in the event that a node is removed to recover, or spread out when a new node is added.
Although this scheme might not work too well, if the number of participants is not uniform, and one partition is overwhelmed, as was the case with How Discord Scaled Elixir to 5,000,000 Concurrent Users implementing a similar scheme
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









