Scaling LiveView for low latency in multiple regions

What backing store do you have? When you start to geolocate your nodes, you’re then changing the latency from User<->Server to User<->"Edge Server"<->Database Server (for example).

It this actually a liveview specific problem? I’d imagine you’d have the latency for every other tool as well. The only thing truely liveview related I can see being problematic in such a setting is the initial double request of http + websocket connection.

Good point, I had considered that but forgot to write it…so yeah, assume at least read replicas in regions too, I guess? Postgres is DB. We’re e-commerce so not write heavy and can eat the latency on puchases etc to a main master.

Maybe not LiveView specific, as yeah, any client-server application will suffer latency, but as LiveView mixes UI and data in what it returns from the server, the experience could be worse than an SPA calling just for data.

Another reason why I feel like it’s a bit special for Elixir, is due to BEAM and OTP being able to be distributed, but I don’t have much experience with that so am unsure if help or hindrance.

Generally payload sizes should at least be comparable, but I can see a difference in e.g. loading a big chunk of items and paginating purely client side, vs. being easy on the initial request, but going back to the server for sorting/filtering and such.

I’m surprised this question didn’t come up earily and I even wondered do elixir devs actually serve app worldwide? Demo-ing liveview on localhost:4000 is not impressive. And I only hear stories about single region cluster.

Ok, real answer: I just use fly.io (alternative: appfleet.com), though they are not cheap, so I have my prod there, and staging on DO’s PaaS (the app platform) where db and app server sit together. The experience so far is that those edge servers are actually valuable, fast and is suitable for liveview. But the edge server ↔ db distance is still tricky. So I read from edge server and response first, then (send self and handle_info) read,merge,write back that result to db. Mostly the former response and the one written to db is consistent. If it’s conflict later, just respond error follow former optimistic ui.

I also plan to add something like Cachex in prod for read (for optimistic ui), and still write back to db. My logic is not near db, it supposes to have legit result in order to write (to db), otherwise, response error or blow up, to maintain data consistency.

For worldwide liveview users with smooth UX/UI, I’d love to hear about other choices as well. I think most folks just target local users, on-prem, or latency expectation that is slightly better than read/write db for every request.

Distribution does not magically solve the problem of a high-latency connection from someone far away; you’ll still need to use e.g. a load-balancer in front (or maybe region-specific DNS settings, or both?) to make sure people try to connect to the server closes to them.

Distribution does however allow people to e.g. be notified of each-other’s changes without having to do a database round-trip, which might be very valuable depending on what you are building.

For the initial page load, LiveView does support that you load it e.g. through a CDN.

And besides this: In general (of course it depends on your particular application) I think it is a good idea for a growing interactive application to decouple the database from your application logic as much as possible; in essence: do not make requests to the DB block usage of the UI, but perform them asynchronously and display the results whenever they become available (even if this takes a second or more). This ensures that the UI experience is smooth even when results take a while to be fetched.

Correct. This is not a LiveView specific problem and it is going to happen on whatever app running between EU<->NZ. The LiveView specifics to this discussion are:

1. The initial request is rendered twice: once with a regular HTTP requests and another one over WebSockets

2. All upcoming live_patch/live_redirect happen on the established connection. This improves UX because we don’t send the layout again nor the browser has to load it again (this is similar to what you get with SPA, Turbolinks, unpoly, etc)

3. LiveView automatically caches and reuses templates in the client - so it sends less data than other server-rendered HTML solutions (similar to what you get with SPA)

4. LiveView runs on WebSockets, which means we don’t need to parse headers, authenticate user on the DB and so on on every request which improves response times (you can get similar with SPA if you are running on WebSockets)

So besides the initial request, LiveView should be helping with the user experience, but the discussion is definitely more general. So it probably makes more sense to open the discussion beyond the context of LiveView.

EDIT: Oh, you can also call liveSocket.enableLatencySim(200) in your browser console to have LiveView simulate latency so you can see which part of your app is not providing proper UX under high latencies.

I wonder can we do something like this on mount to distinguish first connect from reconnect
(It’s more likely there is a reason this can’t be, but i didn’t know):

socket = 
  if get_connect_params(socket)["_mounts"] == 0, do: mark_no_render(socket), else: socket

If the goal is to optimize at this level, I think it is easier to skip the “disconnected render” and render something like “Loading…” (or nothing) instead. You can do so if your pages are private (i.e. they require login) or based on user agent, etc. Although I haven’t really seen a need for this in practice yet.