Livecoding EEx template improvements (twitch livestream)

Thanks @tmbb for the compliments and all of the ideas so far. The changes I did in EEx are indeed a small subset of what we would find in Vampyre or LiveView.

Today I live streamed the implementation of LiveEEx: https://github.com/josevalim/live_eex

The engine does two main things:

• It keeps static and dynamic parts apart
• It tracks when each assign is used and it does not compute such parts if the assigns did not change

The analyze code is relatively straight-forward and all it does is to track whenever an assign is used. However, in order to not change the semantics of the code, we give up the analyze if it sees any variable or lexical command.

This is a different set of optimization compared to Vampyre. Vampyre is about increasing the amount of static parts. Currently I am working on reducing when the dynamic parts are sent. Both are very useful.

Tomorrow we should work on template fingerprinting so we can handle nesting and we will likely work on the Phoenix integration. The whole thing should be less than 400LOC, which is really neat, and this would not be possible if it was not for your initial suggestions on how to structure compiled code!

I just went ahead and implemented fingerprinting now. Here are the docs on why we need it.

I have also improved the analysis code to be more optimal and it is around 110LOC. There is a big comment section explaining how it all works: https://github.com/josevalim/live_eex/blob/3b4a8b727d577d8885f8816ef9adb3d244cb6acf/lib/engine.ex#L399-L511

With the analysis improvements, everything is on 440 LOC, slightly more than I expected but also doing more than I expected, which is quite good for everything it does. The test suite is also really good, so I recommend checking that for any questions.

I have also added support to for-comprehensions to live_eex, which drastically reduces the amount of data sent in for comprehensions, even on regular rendering. Take for example Phoenix’s scaffolded HTML page for index:

<%= for user <- @users do %>
  <tr>
    <td><%= field1 %></td>
    <td><%= field2 %></td>
    <td><%= field3 %></td>
    <td><%= field4 %></td>
    <td><%= link "Show", to: ... %></td>
    <td><%= link "Edit", to: ... %></td>
    <td><%= link "Delete", to: ... %></td>
  </tr>
<% end %>

If you have 10 users, the example above would send the static parts (the tr and td) for each user. Now we just send the static parts once regardless of the number of users. And if you are adding or removing users, the static parts are never sent again. This reduces the data of each initial render and of each update alike.

After all of those optimizations, the rainbow example that Chris showed at ElixirConf is now sending only 1% of the data that it did at the time. Back then it already rendered at 60 fps and reducing the amount of data sent is definitely going to make it more resilient to latency.