Hello everyone! I’d like to understand a bit more about phoenix channels. I watched a video talking about 1 million users in a single room (so i’m assuming its like everybody connects into room:lobby) and it was pretty amazing but made me think about this next scenario. Assuming i have a page with 200 posts and I want to me notified each time when something happens with each of them. I thought of something like
GET /posts → list 200 posts and each of them connects to a socket in the format of post:id so we’d have something like post:1, post:2 and 1 request opens 200 connections 1 per post. If 5000 users access this page we’d have 5000 users listening to post:1, post:2 and so on but we’d have created a total of 1_000_000 connections, right? if the math is right ( I’m not saying it is haha ) with one phoenix app I could only handle 5k users at a time or the way I projected the problem was not right at all?
I would suggest having a full picture of what you are about to achieve and look at how others did it before attempting it and getting stuck in a local sub optima.
For instance checkout this book:
Also I would say batch updates for a lot of posts. i.e. Instead of updates per post, make it update per group or community or room!
Next, group notifications together, instead of sending notifications for each small change, send a single update after significant amount of changes!
For example when I comment in about 10 posts in this forum, I just have to be subscribed to those 10 posts, and not the whole shebang of posts that gets published on the first page or entire site.
That way I can get a notification where I got involved and not the whole site.
Plus I only get notified once for several likes at once instead of filling my inbox with 10 likes for each of those likes I got!
P.S. I tackled this channel creation and update subscription at a much larger scale with several instances running, because it was written in Node. And I was naive at tackling this problem.
When I write my next implementation with Elixir, I don’t want to make those same mistakes. Also Elixir can handle a lot more as it’s multi threaded and distributed by nature.
Lets abstract out the scenario:
- there are n posts and broadcast any changes to users
Rest of the things in the post about sockets, connections needs to be addressed.
- you need only one socket to connect to the server (in this scenario)
- you can multiplex on the socket using phoenix channels
- one channel can subscribe to other topics. This simplifies things because your client need not join 200 channels if there are 200 posts. You can create subscriptions to those topics from channel implementation rather than client joining those topics.
This will reduce to n connections per user.
Browsers impose limits on number of open connections - for example 6 persistent connections per host, 255 web socket connections. This varies for each browser like firefox, chrome, etc.
5000 users can be served with 5000 connections(sockets) - you don’t need to open new socket for each post. Even client can connect to a single channel like they can receive all updates.
If I understood correctly i’d have 5k socket connections and 1 million ‘joined channels’? How many channels can a connection join?
Lets say all connected users receive updates regarding modifications to every post - there is no reason to have post based topics. All the updates can be broadcasted to say common
post_updates channel/topic. At the minimum you will have 1 socket and 1 channel for pushing updates to a user.
I have been trying to find a video, I watched a few months back, ever since I saw your question.
Checkout this talk from ElixirConf 2017. It took me so long to find.
ElixirConf 2017 - Cajoling the Bird: How we squeezed more out of Phoenix PubSub- Simon Zelazny
Thank you so much for the help and effort