Optimistic UIs in LiveView

I fiddled around with existing clocks and references.
I think we need a completely new one for the purpose.

I can’t reuse view.ref because server initiated events would mess up current button references when synchronizing the clocks.

I also can’t reuse underlying transport numbering because keep-alive messages could mess up the clocks.

So I believe, I would have to add the clock to the payload.

Yes, We would need to make sure that the hook is closely tied to the event for two reasons:

• we need client.clock tied to sending event to mark the DOM
• it would be easier to work with if we use the same event name in JS and Elixir (both implementations should do similar thing, but the one from server should eventually overwrite the one client)

So I would even send the event before running the hook.

The clock example will definitely work with what we have in mind. The trick though is that, because you have to update the client, you won’t be using phx-click when you first click on the button, but rather a hook, so you can update the client and then push the event.

Some questions to make sure I understand the solution proposed here. Let’s work on the counter example to have something concrete.
We have two DOM elements: the button that emits click event and the counter.

Where do I use the this.syncPushEvent? Do I attach it to onClick of that button inside mounted section of that hook? Will that prevent sending regular event?
Where do I put the code that updates the counter? in the updated section of the button hook? How do we know we should not update the counter in the meantime?

I think that the solution LiveView currently uses for buttons can work only because the event and state update are for the same DOM element. In case of optimistic UI, we should be able to change multiple parts of the page with a single event.

I was thinking about a different solution using client and server clocks.

1. JS part has the vector clock client.clock. The server has its own, let’s call it server.clock
2. Server increments its own clock every time it sends new state (e.g. an event may originate in the server from PubSub). That part is really important and AFAIU is not part of the current solution with view.ref.
3. I am marking UI elements that I think should be lazily updated e.g: <p phx-update="optimistic"><%= @counter %></p>.
4. I am attaching hook to emitted LiveView event. E.g. <button phx-click="inc" phx-hook="Button">+</button>
5. In the hook, there needs to be a section for running this custom code right after event gets sent to the server:

Hooks.Button = {
  onEvent("inc") { //should be called right after "inc" event is sent
    counter = getElementById("counter")
    mark_updated(counter) //add something like `phx-clock="#{client.clock}"` to the counter
    ... //increment the counter to achieve instant update
  }
}

6. When state update comes from the server, it has the server.clock.

if server.clock > client.clock, do: client.clock = server.clock

// when updating elements
if element.phx-update=="lazy" && server.clock >= element.phx-clock do
  normally_patch_the_DOM()
else
  ignore()
end

I believe that solution is much more general because it takes into consideration both events started by the browser and server.
It can’t replace the current solution for buttons with view.ref because you need to precisely know which client event that button is handling.

Also without separate clocks, you can’t handle optimistic updates originating on the server.

Does that make sense?

The machinery might seem complicated but I wanted it to work on examples like this:

1. User quickly increments counter twice. JS optimistically sets it two. Counter = 2 Client clock = 2.
2. Server processes first event. Sends Counter = 1, Server clock = 1. Since Server clock is lower than client clock, it doesn’t do the update (so the counter doesn’t go back)
3. Server processes the second event. Sends Counter = 2 Server clock = 2. Now the counters are the same so we replace the number 2 with number 2 (everything is fine)

This works even if server-side increments the counter a bunch of times.

I have talked to Chris about an API like this:

1. In a hook, you will be able to call this.syncPushEvent(...)

2. this.syncPushEvent(...) will block all over-the-wire updates to the Hook, until an ack to the syncPushEvent is received. Custom JS code in the hook still runs though

3. Once a reply for the push event is received, we call the hook updated callback, so you can apply the server state to the hook contents

It is basically the mechanism that we use for optimistic UIs in buttons and forms, except we are making it available for hooks too. WDYT?

Haha I might not have been clear enough

Thanks for validation! I’ll check if I am able to dig in into infrastructure enough to propose something meaningful.

I currently imagine the API like this:

• tag parts of the template as “optimistic” to prevent updates with stale values from the server
• allow hooking into JS LiveView events when they fire to do changes to those parts in the template (in the repo I am using phx-hook but I would like to tie optimistic updates to JS LV events instead of DOM lifecycle events)
• some functions to modify the DOM but with setting the time of modification to current event time

Seems like I misunderstood the intro. Seems like counters as examples nowadays automatically mean distributed state to me

FWIW, LiveView already does this internally. For example, imagine you have three buttons on the page. Once you click them, they change their text to “Processing…”. Processing each button takes 1 second. If you click all 3 of them one right after the other, the first response will come in t=1, then t=2, then t=3.

if LiveView did not track the events, once t=1 arrived, all buttons would revert to their original text, but it doesn’t.

So maybe we can generalize this a bit so people can also leverage this information in hooks or similar, but the whole infrastructure is already there. This may be possible, I think it is a great idea!

Not really. The goal with optimistic UI is to change the client as a way to optimistically guess what the server will return. There is no state is really. The changes you do on the client will be discarded as soon as the ack is received.

You’ll lose that as soon as your client becomes stateful for the state you also deal with on the server.

This is exactly the reason, why liveview doesn’t synchronize state at all. There’s state on the server and there’s markup derived from this state. The markup is sent to the client and replaces existing markup. There’s the fancy markup diffing, where the client does indeed hold some amount of state (the static parts), but everything on the client is replaced in the case of a reconnect.

This basically boils down to an CRDT implementation. There’s e.g. automerge for JS, which implements crdts for json data and also has existing tooling for synchronising data via network connections. The “problem” for an elixir based solution is that you need node on the server or you need to re-implement the automerge crdt protocol in elixir. I’ve experimented with automerge from client to client proxied through phoenix channels in the past and it’s interesting tech, but also complex.

Thanks for your input! If I understand you correctly, you propose to write the core event handling logic in JS so that you can reuse it both on the server and client and then use channels to synchronize the states.

It might work but LiveView gives me much more. The server LiveView process is a single source of truth and I loose that if I use plain channels. The state synchronization is very hard to do efficiently so I really appreciate how great LiveView is in reducing the payload size between client and server.

LiveView also gives me entire lifecycle in case of dropped connections. Doing all that using channels seems like a big hurdle that would end up in a half reimplementation of LiveView.

I might be greatly underestimating the task of adding event counters to client and server side of LiveView but after we have that, optimistic UI algorithms boils down to:

if server_clock >= client_clock do
  patch the dom as usual because server is always right
else
  don't patch the dom because user clicked/typed a bunch of things so there are still events coming from the server
end

It might need a separate “client modification timestamp” for each “gap” in the template and some JS functions to set that stamp when you modify those gaps on the client side. But it still seems like much less work than building custom solutions on bare channels.