54) ElixirConf US 2018 – Closing Keynote – Chris McCord

[axelson]

ElixirConf US 2018 – Closing Keynote – Chris McCord (@chrismccord)

All talks thread:

0) ElixirConf US 2018 Talks List

Hey everyone! I hope everyone enjoyed ElixirConf! I know that I did. In order to facilitate and continue the discussion (especially since YouTube comments are disabled) we are going to post a new video discussion thread every day (edit: changing to 3 video discussion threads a day) so that we as the community can give some attention to each of the conference talks. However, since the keynotes will definitely get plenty of discussion we will post all of those at once. If you don’t want to miss a…

[AstonJ]

Awesome keynote Chris!

Loved hearing about LiveView… and this bit made the hairs on the back of my neck stand up

“Challenge the status quo of what is possible”

“We have this platform that can advance the state of the art”

“The future is ours to build”

[tme_317]

Although we’ve seen similar patterns before (i.e. drab) LiveView still totally blew me away and I’ve rewatched the video 3 times now. Now tomorrow morning I get to write tons of React form validation code and repeating the same/similar code in my changeset validations on the server and dealing with all the plumbing… while thinking how much easier it would be with LiveView with no perceptible performance degradation.

The way it was presented it seems like you could make LiveView “components” (stateful and stateless (not needing a GenServer?)) and hook them together like a React component tree. It seems weird to say but maybe we could have “JSX-like” syntax inside the EEx LiveView components which could call other LiveView module’s render functions with props (assigns) and children. All the data would be accessible server-side so less need all the JSON/GQL requests. It seems like that should work… and it could be the right choice for many (not all of course) project types.

Unbelievably incredible job @chrismccord just can’t wait until it is released for experimentation!!

[tmbb]

@tme_317

while thinking how much easier it would be with LiveView with no perceptible performance degradation.

You can probably get something pretty similar to LiveView reuseable widgets right now by:

1. Setting up a channel
2. Hooking up some JS to events in your forms (maybe using custom attributes)
3. Rerender the form on the server
4. Send the HTML back to the client and diff it with the JS morphdom library.
5. (Bonus) have a deterministic ID for the resource so that different cliente can edit it, and you have a (bad) collaborative realtime resource editor (with last-write-wins semantics, which is what makes it bad)

[tme_317]

Hmmm… interesting thanks @tmbb. Definitely your ideas would work while we wait for LiveView.

Since I have already invested several months learning and building a working React-based system (and have no offline requirements) I think I will look into channels to more efficiently do async field validations… at least those that are more complex than simple regexes. I haven’t had a need for phoenix channels to this point and this seems like a good opportunity to learn it finally!

[tmbb]

@tme_317

Definitely your ideas would work while we wait for LiveView.

From the talk, I believe that “my” ideas are basically the same as the ones behind LiveView (although I’ve certainly had them independently). The main advantage of LiveView over something custom like what I describe here is the existence of a widely accepted pattern which user can follow. But by all means experiment with these kinds of solutions yourself!

[brightball]

This really looks great and I’m excited to try it out. My biggest question is about how this will affect Drab? Will the projects consolidate?

[sandymatt]

Similarly, how does this relate to the Texas Server-side Virtual DOM library? The ElixirConf video was pretty good on that.

All these server-side implementations of client-side js apps are appearing quickly. Shows how well put together phoenix is.

[kanonk]

Fantastic video, really excited about this, can’t wait for it to be released. Great work!

[tmbb]

@josevalim

I believe the diffing of input tags could be done on both sides though - it is just a matter of sending enough information to the client, structured or not, for it to happen.

Imagine the following sequence of events (imagine a non-trivial latency):

1. Client sends an input field
2. The server validates the input field and sends the form back to the client
3. The client edits (and sends) the new input field
4. The client receives the 1st diff from the server

In step 5, the client must be smart enough not to undo the changes in the input field (there are many ways of dealing with latency, you just can’t paste the diff from the server while ignoring that it’s an input tag that might have been changed), preserve caret position and all that. This requires some level of “smartness” on the client. I think that if you’re having such a smart client, you should move most of the diffing work there.

But yeah, if you version your diffs with a monotonic sequence, use operational transformations or a CRDT or something like that you can achieve the same results. It just seems simpler to let the client do most of the job.

In fact, I don’t know if morphdom handles the sequence of steps above gracefully. The tests we’ve seen in the video show almost no latency, which allows us to ignore the fact that the client-server connection is actually a distributed system. I’d like to see a demo with realistic worst-case latencies (~ 300-500ms)

[josevalim]

Yes, Texas does the diffing on the server, Phoenix does it on the client. I believe the diffing of input tags could be done on both sides though - it is just a matter of sending enough information to the client, structured or not, for it to happen.

Dan, Chris and I also had a conversation between their talks and I think it is possible to at least reduce the amount of data LiveView currently sends to the client. Today it sends the whole template but they can likely extract the static parts out during compilation time. There are many possible optimizations available on both projects.

[thousandsofthem]

Hmm, why almost no one discusses Telemetry stuff? Not-so-trendy but still very useful and improves the whole ecosystem, making it more production-friendly.

[tmbb]

@thousandsofthem

Hmm, why almost no one discusses Telemetry stuff? Not-so-trendy but still very useful and improves the whole ecosystem, making it more production-friendly.

Maybe people are more interested in improving the development complexity in there own app instead of something abstract like “improves performance monitoring for the whole ecosystem”? I certainly am

[tmbb]

@josevalim

Today it sends the whole template but they can likely extract the static parts out during compilation time. There are many possible optimizations available on both projects.

This is where gutting and extracting parts of the Drab EEx engine might be helpful. From what I remember it’s quite efficient at updating only the minimal tree branches that change. The implementation is more complex than just “render everything and send it down the pipe”. Rendering everything is usually very efficient, because IOLists reuse static parts of the template, it’s the “send it down the pipe” that could be optimized.

Another approach, which I suggest is the following:

• cross-compile the EEx template into a javscript function which takes the dynamic substrigs as arguments (not the arguments to the template function!). For example:

Hello <%= @username %>! How are you? It's been <%= @idle_days %> since we've seen you!

compiles into:

function renderFromDynamic(dynamic) {
  var fullString = "Hello "
    .concat(dynamic[0])
    .concat("! How are you? It's been ")
    .concat(dynamic[1])
    .concat(" since we've seen you!");

  return fullString;
}

At the same time, compile the EEx template into something that only generates the (already escaped) dynamic strings. From what I knwo about EEx templates, this seems very easy. In the case above, compile it into something like:

def render_dynamic_parts(conn, assigns, other_args) do
   [~e(<%= @user %>), ~e(<%= @idle_time %>)]
end

Then, each time the state changes, send only the dynamic parts to the client (s a JSON array of strings; JSON arrays are reasonably compact), and have the above Javascript function above render the full string on the client and diff the result with morphdom or something else.

This has a number of advantages:

1. You can use normal EEx templates (no need to parse the EEx template with something like Floki, like Drab does; from the server’s point of view it’s all strings)
2. The server doesn’t deal with the DOM, which is slow and needs to be serialized (let the browser deal with that; it has to deal with it anyway), only with IOLists, which are fast
3. You save bandwidth because you only send the dynamic parts

[blatyo]

@thousandsofthem

Hmm, why almost no one discusses Telemetry stuff? Not-so-trendy but still very useful and improves the whole ecosystem, making it more production-friendly.

I’m very excited about this. I’m interested in building integration with this for conduit. I think maybe it’s more library authors that are interested in this than individual users right now. I suspect once it’s more widely used in the community, people will start opening issues/PRs to their favorite projects to add support.

[chrismccord]

The simplest approach that we take currently is the client’s input with focus wins, and we never overwrite in such cases. There are definitely areas for us to explore here. Under extreme latency this approach would potentially update blurred inputs with bad values b/w the client push, which would recover but wouldn’t be ideal UX. Some OT or similar approach could work, or at the very least we can track all inputs on a phx_change and never overwrite the inputs until we get an acknowledgement.

Wrt to server diff/vdom optimizations, it’s definitely planned but our first steps are to prove out the user land features. Everything else “just” becomes an optimization story which we can improve as we go.

[tmbb]

@chrismccord

The simplest approach that we take currently is the client’s input with focus wins, and we never overwrite in such cases.

I agree with this. It’s very rare that the server will want to overwrite user input, and if it does, you are in a distributed system and should definitely be using OT.

@chrismccord

There are definitely areas for us to explore here. Under extreme latency this approach would potentially update blurred inputs with bad values b/w the client push, which would recover but wouldn’t be ideal UX.

I agree completely. It’s acceptable but far from optimum.

@chrismccord

or at the very least we can track all inputs on a phx_change and never overwrite the inputs until we get an acknowledgement.

Yeah, waiting for acknowledgement to overwrite input fields looks like the way to go. Waiting for some roundtrips to overwrite is probably always better than messing with the cursor position by overwriting what the user is typing.

[grych]

@brightball

My biggest question is about how this will affect Drab?

I am going to stop developing Drab, as it is not worth to duplicate the same functionality, and LiveView covers most of the areas which Drab does (Commanders, Living Assigns).

[grych]

@tmbb

This is where gutting and extracting parts of the Drab EEx engine might be helpful. From what I remember it’s quite efficient at updating only the minimal tree branches that change. The implementation is more complex than just “render everything and send it down the pipe”.

True.
It is actually modifying the template during the compilation, adding its own markers to the DOM tree and saves which assigns are related to which markers. Thanks to this, it can send only the small parts of the page to be updated.

[hilko]

Would love to see your work on Drab, and/or the lessons you learned from it, applied to LiveView though! Based on the keynote the latter is still very much in (early stages of?) development.