Phoenix Channels and the Presence module are very helpful and do a splendid job with tracking user activity in Channels. For this example, let’s call channels - Rooms.
To get the total users connected to a Room (in a typical Presence setup), we would use something like:
MyAppWeb.Presence.list("room:<id>") |> Map.keys() |> length()
Performing the above operation for single room or up to a few hundred rooms would not be very expensive.
Let’s say we have 10k rooms and we want to sort the rooms based on the number of users in each room. And the goal is to display only the top 10 most active rooms. What are approaches I can take to achieve this?
I’d try to count rooms’ users in parallel using Flow — flow v1.1.0, if I understand its purpose right 
Thanks for your reply @patrickdm. While Flow will replace the Enum/Stream module and process in parallel using Genstages, will it be feasible to perform this operation on every request to this endpoint when there are a large (10-50k) number of rooms?
Do you see any other approach to solve this?
I’m sure I cannot answer that (apart of guessing it depends on requests’ frequency) 
sorry -have no real experience on that kind of problems, yet.-
about other ideas … maybe use a background job to count and cache the most crowded 10 rooms, maybe broadcasting the list… once in a while … just a wild guess (uneducated)
I’m not experienced too, but you may look at separate GenServer with some map like room => quantity and two call’s add and remove, with reply of 10 top rooms. Actually state may be different, for example it could be an ETS table, or something else like Prioqueue, or anything else which better suits for your goal. It shouldn’t be a bottleneck, as computing seems trivial.
Hey @dmitrykleymenov, I was thinking of something similar. Using a Genserver to poll Presence every ~15 seconds, which then updates the data in ETS (as an ordered_set) so it’s more performant in terms of reads.
Yeah, it’s more like a direction to dig. The perfect solution depends on your exact realization. You may update it in some interval(as you mentioned), or on every join/leave room, broadcasting new top-10(if it’s necessary for live-update, i thought it’s the case, actually).
In my case, it’s just an API endpoint that returns the top 10 out of thousands of rooms. So every time the endpoint is hit, the top 10 rooms are fetched.
Broadcasting on every join/leave might be overkill I guess, but it did cross my mind.
I think cast a message(without broadcasting) to that “members” server on every join/leave would be effective and Elixir-style, since it uses only deltas of joining/leaving users, instead of computating through all thousands of channels every time. And you’ll have actual information in every moment. Even if all users of all rooms changed during that 15 second(which is the worst possible scenario for that solution compare to asking Presence), in my opinion, it could be as fast as asking presence. But i’m not sure Just my vision of problem. It depends on expectations about frequency of rooms in/out activity.
3 Likes
