54) ElixirConf US 2018 – Closing Keynote – Chris McCord

I take this back. It is NOT very easy… Chunks like <% x = ... %> require compiling the template into a {:__block__, meta, expressions}, which forces us to deal with variable assignments which might possibly overwrite each other. To split the template into static and dynamic parts it looks like one has to (at least) convert Elixir blocks with variable assignment into single static assignment (i.e. almost the same as compiling macroexpanded Elixir into Erlang). Then it becomes easier to factor variable assignments outside the dynamic strings. I feel like I’m derailing this thread a little, and I might start a new one to discuss these matters.

Instead of choosing LiveView or React, would it be possible to choose LiveView and React (or Mithril, Svelte, Vue, etc.)?

Would an umbrella app be the only way to go?

1 Like

If you mean on the same page, I would think that react and liveview would collide, and to solve it you would have to wire react to reload each time the html was overwritten.

If you did not mean on the same page, you are not limited to any of them. You can have both serverside rendered (ssr) pages, ssr pages with LiveView, and pages with react.

1 Like

@malloryerik - though LiveView doesn’t fit nicely with client side frameworks that render HTML, I’ve also been looking for a sort of middle-ground where I can still use my React components. I came up with something similar to LiveView that renders JSON instead of HTML. On the client side, it takes the JSON output + a function to update and wraps it in a React Provider. Then my other components can use a React Consumer to access the data and update it. It’s basically a very poor man’s server-side redux. It’s a PoC at this point, but I’d love to get any thoughts/feedback.

Thanks!

1 Like

Drab, however, supports more things, such as interactions with API’s on the frontend and more. Live seems more like Drab’s living assigns things and controlling/owning the entire DOM, so Drab definitely still has a place! :slight_smile:

4 Likes

This is true, actually Drab is quite mature, because it’s been born in the long discussions in this forum.

  • small reads and updates of DOM (Drab.Element, Drab.Query)
  • quick access to the browser-specific attributes, like a local timezone (Drab.Browser)
  • Drab.Modal, which is IMO a killer feature as it actually hold on processing and waits for the user input
  • Drab.Waiter, which does the same but more general, without a modal
  • and, last but not least, ability to calculate using browser (exec_js(socket, "2+2"))

So after reconsideration I am going to move Drab to the v1.0, as it is just a few steps ahead, and a lot of code for this is already done. I hate wasting good code. The project will be supported (bug fixes) as long as people are using it, but no new big features will be developed by now (sorry @OvermindDL1, no token via cookie, as I want to wait to see how Chris will cover this, but we may think about building our own transport as it would be easier since phx1.4).

Then I will take a break and will see what will born as a LiveView. If, as I assume, it will be a huge success, I will give up Drab.Live and Commanders and adopt rest of the Drab features to the LiveView, using its event handler API (phx-click attributes). Which accually I like more than mine, I probably overcomplicated stuff with Commanders, but have some concerns if it can cover some cases.

21 Likes

A kind of server-side redux for JSON or LiveView / Drab for JSON sounds great.

1 Like

Well I’m definitely making a lot of use of Drab for sure. ^.^

I still like commanders, makes it easy to make reuseable little widgets (like a notification widget for example) on many pages but that commander just controls that tiny section of it. :slight_smile:

3 Likes

I’m not sure I want redux on the server. Servers have been handling state without such patterns with great productivity for a long time. I can only speak to the texas impl. but state is not something texas cares about because server-side is already great with tons of solutions for handling state.

1 Like

I think texas can do the “smart diff” on nested dynamic templates - I’m not familiar enough with liveview to speak to it, but with texas all you’d have to do is attach an id to the dynamically generated “partial/component/whatever”

For instance <div data-texas="conversation" data-texas-id="123"> ... </div> could be generated dynamically and the only thing you would have to do with texas is broadcast a message that looks something like "conversation:123" (psuedocode, not real impl) and subscribers to that message could then call a view rendering function that pattern matches to see if an texas-is was given, if so, only render out the dynamically generated partial/component for the given id

edit: update with example

Oh, totally. I meant to say that we can’t do smart diffs with .eex (or a subset thereof) unless we introduce parsing + some restrictions which I assume would put us somewhere close to .tex. :slight_smile:

makes sense, sorry I’m glossing through all these conversation and I’m late to the party so I probably missed that context

You can’t do smart HTML diffs, of course, but you can do some smart textual diffs given two restrictions:

  1. Change the EEx engine semantics so that each <%= ... %> block runs in its own scope. This means you can’t have <% a = 1 %><%= a %>.

  2. Add a new block type, say, <%| ... %>, which runs its contents at compile time and inserts the result

  3. Write a bunch of “compile-time” combinators (for example, a compile-time form_for(), which returns another template with these restrictions. These combinators could be macros, but it’s better to make them functions that return quoted expressions (like in nimble_parsec).

This way, you can tease out the static and dynamic parts of a template at compile time, as long as the level of nesting is bounded and you’ve written suitable widgets. That allows you to separate the dynamic parts of the template at compile-time.

I’m working on a prototype, and it’s pretty cool.

If you then use morphdom on the client, you can just reconstruct the string on the client, and diff it into the existing DOM.

EDIT: This can even be used to optimize the “normal” string-based templates (by compiling into static binaries as much of the template as possible).

1 Like

Then it is no longer EEx. :slight_smile:

I don’t think we need this? If we make form_for a macro AND we make the “smart diff” mechanism work on any safe IO data, then we don’t need precompilation, as long as form_for emits data in the safe io shape.

2 Likes

If you’re being that strict, then yeah. I still think that one could use smart diff on “normal” eex templates by transforming the resulting quoted expression into “erlang-like” elixir (with no variable rebindings and stuff like that). But that requires reimplementing most of the Elixir compiler, which although fun doesn’t seem easy.

Yeah, you’re right… I don’t know if Plug.HTML.Engine can make use of this oprimization right now, but if it can’t it’d be a cool pull request. It would make “normal” templates more eficient.

Besides form_for, most of the widgets in Phoenix.HTML could be optimized by reimplementing them as macros. I should benchmark it.

1 Like

Yeha, I think the goal is not break the model we have today, and that would definitely break it.

Exactly. There are a lot of changes here we can just push to the default template in general that will make it more efficient too and better for the diffing.

Regarding form_for, one approach that Chris and I thought was to render it like this:

<%= f = form_for @changeset, Routes.some_path(...) %>
  <label>...</label>
  <%= input_for f, :name %>
  <%= error_tag f, :name %>
</form>

In this approach we remove the nesting and it will work as long as we implement Phoenix.HTML.Safe for Phoenix.HTML.FormData. But it goes against your first rule. :slight_smile:

1 Like

I don’t know if it is helpful, but this is how Drab does it. In compile time:

  • it has its own EEx engine
  • during compilation the Drab EEx, while handle_expr, it is checking if there are any assigns inside the expression
  • if there are assigns, it is injecting the special attribute (drab-ampere), to the parent tag; if there is no parent tag, it is surrounding the expression with the <span>
  • injecting the attribute also checks if the assign is in the tag body, or in the attribute
  • there is a special mark, <%/ %> which marks expressions which should not be ‘drabbed’
  • at the end, it collects all amperes with the corresponding assigns; this is how it knows where to change if you update the assign in a runtime
  • assigns are stored in the frontend, Drab injects javascripts like window.__drab.assigns.myassigns = <%= @myassign%> to the template

In a runtime:

  • you render the template as usual, with the controller
  • on connect, Drab gets the current assign values from the frontend and cache it in the genserver
  • on any assigns change (called poke), Drab re-renders the template on the backend; because it know exactly where in the template the assign you are changing is begin used, it gets the value of the tag body (or the attribute value) with Floki
  • then it sends the JS back to the browser - it is a simple update of the specific tags only
  • it also sends back the new assign value to the browser - this is important, because in case of disconnect the Drab genserver is killed

There are also corner cases, like a special treatment for @conn, or keeping the csrf token in case you update the whole form, or detecting which ampere to update in case they are nested.

All the above looks complicated, but this was the only what I could invent with the assumptions that:

  • you do not require many changes to the existing phoenix project: in this case the only change is the extension from *.eex to *drab
  • it must be beginner friendly (to convince beginners who are scared of Ajax and JS)
  • it must work as expected, be intuitive (this is why assign values must survive disconnection)
  • all must be archived with external library only, without any changes to Phoenix (as they are not very welcomed)
8 Likes

Besides going against my first rule, this approach is conceptually very weird, at least to me. You have a widget for the opening tag but you must close it manually, in a way that’s completely opaque to someone who looks at the code. I think I prefer turning form_for(form, action, options, fun) into a macro which calls fun.(form) at compile time.

The main problem is that form can be anything that implements a protocol, so it would have to be converted into %FormData{} by all inner widgets. That’s very inefficient (although still the right choice if we want to optimize for bandwidth at all costs!).

The problem is that with my model (respecting the first rule) you can’t precompute the conversion to %FormData{} at runtime! So you’d have to perform that conversion outside the template (this is a limitation of my apparoach I didn’t think of earlier…)

There might be a way out that avoids my rule, though (I’ll get very technical here). The semantics of variable rebinding and its scope is what got us into this mess. In an EEx template, we must respect rebinding semantics, which means an EEx template is compiled to a series of nested lists with blocks inside the lists, which is kinda messy to refactor automatically. For example, if you compile this template:

<%= a = 1 %>
<%= a %>
<%= a = 2 %>
<%= a %>

with the default phoenix engine, you get this:

{:safe,
 [
   (
     tmp1 = [
       (
         tmp1 = [
           (
             tmp1 = [
               (
                 tmp1 = ""

                 [
                   tmp1
                   | case(a = 1) do
                       {:safe, data} ->
                         data

                       bin when is_binary(bin) ->
                         Plug.HTML.html_escape_to_iodata(bin)

                       other ->
                         Phoenix.HTML.Safe.to_iodata(other)
                     end
                 ]
               )
               | "\n"
             ]

             [
               tmp1
               | case(a) do
                   {:safe, data} ->
                     data

                   bin when is_binary(bin) ->
                     Plug.HTML.html_escape_to_iodata(bin)

                   other ->
                     Phoenix.HTML.Safe.to_iodata(other)
                 end
             ]
           )
           | "\n"
         ]

         [
           tmp1
           | case(a = 2) do
               {:safe, data} ->
                 data

               bin when is_binary(bin) ->
                 Plug.HTML.html_escape_to_iodata(bin)

               other ->
                 Phoenix.HTML.Safe.to_iodata(other)
             end
         ]
       )
       | "\n"
     ]

     [
       tmp1
       | case(a) do
           {:safe, data} ->
             data

           bin when is_binary(bin) ->
             Plug.HTML.html_escape_to_iodata(bin)

           other ->
             Phoenix.HTML.Safe.to_iodata(other)
         end
     ]
   )
   | "\n"
 ]}

(those not experienced in looking at printed macro output, remember that parentheses on their own lines are block delimiters in Elixir)

With my simplified semantics, without variable rebinding, I can compile the template into something like this (I’m cheating a little by removing the {:safe, ...} tuple, but you get the idea.

[
  "",
  [
    case(a = 1) do
      {:safe, data} ->
        data

      bin when is_binary(bin) ->
        Plug.HTML.html_escape_to_iodata(bin)

      other ->
        Phoenix.HTML.Safe.to_iodata(other)
    end
  ],
  "\n",
  [
    case(a) do
      {:safe, data} ->
        data

      bin when is_binary(bin) ->
        Plug.HTML.html_escape_to_iodata(bin)

      other ->
        Phoenix.HTML.Safe.to_iodata(other)
    end
  ],
  "\n",
  [
    case(a = 2) do
      {:safe, data} ->
        data

      bin when is_binary(bin) ->
        Plug.HTML.html_escape_to_iodata(bin)

      other ->
        Phoenix.HTML.Safe.to_iodata(other)
    end
  ],
  "\n",
  [
    case(a) do
      {:safe, data} ->
        data

      bin when is_binary(bin) ->
        Plug.HTML.html_escape_to_iodata(bin)

      other ->
        Phoenix.HTML.Safe.to_iodata(other)
    end
  ],
  "\n"
]

You can see that I was able to optimize the template into a list where alternate elements are either static or dynamic. That is very hard to do with the default HTML template…

I’ve thought about trying to build lists as I walk the EEx file. The output of a EEx engine doesn’t need to be a quoted expression representing the template, which means I can thread a context as I parse the source and then try to output something sensible from the context, but so far I’ve been having troubles. No matter how you look at it, the output of an EEx template seems to require a deeply nested tree of blocks with many instances of variable rebinding.

This is a problem Elixir already has to solve when compiling to Erlang (there are no variable rebindings in Erlang). To compile to Erlang, Elixir uses something called Static Single Assignment, in which something like this:

a = 1
a = a + 1
a = a + 2

becomes:

_a@0 = 1
_a@1 = _a@0 + 2
_a@2 = _a@1 + 3

If we could replace the reassigned variables using Static Single Assignment, we could probably turn the AST inside out, and move all variable calculations outside the tree and collapse the tree into a simple flat list. I’ve been avoiding going that way, because that requires an intimate knowledge of the core elixir semantics (namely, it requires the programmer to know all places where variables can be bound). But besides that, I can’t see any way of simplifying the AST returned by the EEx engine and separating the static from the dynamic parts.

Thanks, this is very interesting! How do you handle an injected <span> that may appear multiple times? Imagine on the first render you have 3 comments, then there is a new comment, how would you track this on Drab’s side and send it to the browser? For a reference, the approach I have in mind would render all of the comments again and send it to the browser for diffing.

Also, how do you sync assigns in case of disconnect+reconnect? Because the data on the server may be more recent, so you can’t just trust the assigns the client sends?

4 Likes

Yes, I agree. :slight_smile: Just bringing options to discuss!

Well, since we have deprecated var as a function call, a variable in EEx is either a variable being invoked (which means it has to be previously defined) OR it is being defined. If a variable is being invoked, then we can’t do the transformation you propose. If the variable is being defined, then we may be able to do it but how frequently do we define variables in templates?

So can’t we flatten it as long as there are no variables? It should be safe for Phoenix because the only variable we have is assigns and most of the assigns accesses are hidden behind @access, which means it should just bypass the optimization you propose.

1 Like