Here’s the story how one of the world’s first production deployments of LiveView came to be - and how trying to improve it almost caused a political party in Germany to cancel their convention.
I wrote this post just a few days after the event took place. As annoying as it was, it was a good teachable moment. And soon I’ll write an update with a tutorial on how to scale to 5,000 concurrent LiveView users on a single VPS
In regards to point #1, it seems like the argument here is that instead of one process querying the DB and sending the result to the other processes as a PubSub payload, it’s better to have the process send a message without payload, and then each process makes its own DB query. Is this always true? I use payloads in my PubSub broadcasts sometimes to minimize DB queries; now I’m wondering if this was a mistake.
Large payloads over PubSub by themselves are just fine. We actually broadcast entire ets tables of presence information when we replicate data for new nodes, so by itself this isn’t a problem
This is the key takeaway here. Publishing events at scale must consider downstream subscribers. In this case publishing the entire user list for any individual user change was the mistake, rather than deltas of joins/leaves (or presence)
We won’t send anything on the wire if there is an empty diff, so I’d need to know more about this particular case.
In my case: No. The state is being requested from a GenServer and not a DB.
I’m sorry it’s being perceived that way - and I’ve heard the same from a couple of commenters online. I understand how the title could be misread as disparaging the use of LiveView which was absolutely not my intention (which is hopefully very obvious when you read the article).
The problem is that if there is a change in the assigns which doesn’t result in a change in the template (because the assign isn’t used in the template), it still sends an update to the browser.
That’s a very interesting point, thank you Maybe the takeaway shouldn’t be a general discouragement of large payloads but rather to be extra mindful whether they are necessary or not.
This was definitely my thought when i read the diagnosis of the crash. Was there a design story here that really called for every single update to be printed in real time to begin with? What would an admin user even do with such information? Seems like if you are sending so much data that even the browser couldn’t keep up, what was the human user going to do?