Hi all, I have a design question about implementing a distributed chat server. I’ll start by explaning my use case, and why I find it difficult.

Basically I’m trying to build a simple Discord for our special need. For those who haven’t used Discord before, Discord is a text & voice chat app which uses WebSocket for text chat and WebRTC for voice chat. Users can join a channel (i.e. a chat room) and start talking to each other.

In order for the users in a channel to hear each other, the server needs to maintain WebRTC connections to all the users in the channel, and forward the WebRTC media packets from a user to all the other users in the channel. For this reason, I imagine it’d be necessary (at least much more efficient) to have all the users in a channel connected to the same server, so that the voice session state (e.g. WebRTC connections) lives in a single process.

But I can’t seem to see an easy way to gaurantee that users in the same channel are connected to the same server when we are running more than one servers. I considered using stick sessions, but after some research it turned out only vanilla HAProxy supports sticky sessions based on URL params, and all other load balancers/reverse proxies/etc. only supports sticky sessions based on HTTP headers or cookies. The problem is, WebSocket requests can’t have custom headers.

I’m curious about how Phoenix Channels are implemented. Is it gauranteed that all connections to a topic is to the same server? If not, how does Phoenix solve the problem of having to forward messages of one user on a topic to the other users on the topic? Does it need to forward the messages first between different servers?