Benchmarks

I have uploaded an example project, so that I can run “more realistic” benchmarks. You can find the repo here. The benchmarks below are the result of rendering the form generated by the phoenix generators for a dummy User resource. This means the template is probably representative of “real world” templates.

You can run these benchmarks yourselfby running the following command in the repo above.

mix run benchee/main.exs

The results are:

Benchmarking phoenix...
Benchmarking vampyre...

Name              ips        average  deviation         median         99th %
vampyre      123.41 K        8.10 μs   ±231.37%           7 μs          25 μs
phoenix       10.13 K       98.69 μs    ±20.93%          95 μs      180.18 μs

Comparison:
vampyre      123.41 K
phoenix       10.13 K - 12.18x slower

This shows that in the case of forms, Vampyre templates are about 12x faster than the default phoenix templates. The absolute time differences are tiny, of course, because we’re talking about microseconds, but the improvement is quite impressive.

I still haven’t implemented all the HTML widgets implemented by phoenix_html. In fact, I have implemented just enough to be able to render the output of the Phoenix generators, but most of the rest can be implemented on top of what I already have. This is not one of those cases where “the code is simple and fast because implementing the last 10% takes the last 90% of the code/work”, because the work my widgets do it at compile-time. Although my incomplete implementation of the HTML widgets is already more complex than the one in phoenix_html (in terms of lines of code, and concepts I introduce throughout the code base) the complexity happens at compile time with the goal of making things fast at runtime.

Am I doing something dirty to get these benchmarks?

Yes. The implementation of form_for that makes it fast inlines a function call by introducing a hygienic variable in the global scope of the template. This doesn’t change the semantics of the template too much because the variable is inaccessible to the user-written code. But it is an issue if you nest form_for inside another form_for. This is not valid HTML but there might be reasons why you’d want to do that. In case the above isn’t scary enough, I reproduce the code here:

  def make_form_for(form_data, action, options, fun) do
    {:fn, _,
     [
       {:->, _,
        [
          [arg],
          body
        ]}
     ]} = fun

    validate_fun_arg(arg)
    hygienic_var = hygienize_var(arg)
    substituted_body = substitute(body, arg, hygienic_var)

    open_form = Tag.make_form_tag(action, options)

    hygienic_assignment =
      quote do
        unquote(hygienic_var) = FormData.to_form(unquote(form_data), unquote(options))
      end

    close_form = Segment.static("</form>")

    contents = [
      open_form,
      Segment.support(hygienic_assignment),
      substituted_body,
      close_form
    ]

    Segment.container(:lists.flatten(contents))
  end

I’m pattern matching on the AST of the function in the form to get the variable given as argument, an then, in the bpdy of the function, I substitute the old variable by a new one, and dump the body of the function into the global scope of the template. This avoids a function call and flattens the segments list, so that I can merge them together.

I think I can the issue with nested forms by doing even dirtier things with the process dictionary (I can safely assume that the template will be compiled by a single process) and using the variable’s :counter in the metadata to make variables even more distinct from each other.

What’s missing

I still haven’t implemented all HTML widgets. Also, I haven’t even implemented all widgets correctly. Because I’m aggressively optimizing everything, there is some heavy-duty case analysis on the widget-generating macros/functions. For example, tag(:input, name: "user[name]", id: "user_name") takes advantage of the fact that all attrbutes are static to render everything into a static binary (which can be merged with the previous or next binaries). However, if we change the above to tag(:input, name: @name, id: "user_name"), the expression will be rendered into a static part, followed by a dynamic part, followed by another static part. If the user writes instead tag(name, attrs), then this is a case I don’t even handle yet!

So I have yet to implement the “maximally dynamic” versions of my widgets. In such cases I might just fall back to Phoenix’s implementation, but there are still some performance optimizations to be had. For example, even if both the tag name and the attributes are purely dynamic, we know the rendered value starts with "<" and ends with ">", which can be merged into the previous binaries.

Is this worth it?

To render normal templates, I don’t know. As I said above, the relative differences in performance are impressive (I wasn’t expecting such a difference), but rendering templates is already so fast that maybe it doesn’t really matter. And all the complexity can introduce bugs, security vulnerabilities, etc.

To use with something like Drab or LiveView, or something like that, where being able to siolate the static from the dynamic parts of the templates is critical (to minimize sending data over the network), this is definitely worth it.

Even if for that reason alone, I plan on continuing to work on Vampyre until it has feature-parity with phoenix_html. Vampyre is already a drop-in replacement for phoenix_html (by changing two lines of code in your project you magically get Vampyre templates for free), and if the user decides to use Vampyre templates for something like LiveView or Drab, then it makes sense to use Vampyre templates to render the “normal” views.

It will take some time, of course, because I’m not only implementing template renderers. I’m actually implementing an optimizing compiler for template renderers, and the default widgets need to play well with the compiler.This means that everything is about 5x harder to write than the naïve versions in phoenix_html. It gets easier as I get further away, because I can implement some widgets on top of other widgets. For example, I’m implementing the form_for widget on top of the tag widget.

EDIT: I believe I’m doing everything right when defining the renderer functions for both Vampyre and Phoenix templates. I’ve copied the approach taken by Phoenix itself, so I guess I’m not being unfair to the Phoenix templates.

After the silence, some news!

(@josevalim, there are almost the news you’ve been waiting for regarding expansion of macros)

I can finally expand Elixir macros inline inside the templates and optimize the resulting template by merging the static binaries together. I still can’t compile my new templates into executable Elixir code but it doesn’t pose any hard problems.

I’ve yet again changed the template format and the nomenclature. Templates are now composed of segments. A segment is either static text or a quoted expression represented by <%= ... %> (dynamic segment), <% ... %> (dynamic segment with no output) or <%/ ... > (fixed segment).

Segments are now represented as:

{segment_tag, {contents, metadata}}

As you can see, templates and parts of templates are represented by a 2-tuple nested inside another 2-tuple. The slight change from the previous post is because it’s useful to be able to store metadata in the segments for error reporting. Why nested 2-tuples instead of a 3-tuple? It’s because 2-tuples are represented as themselves in Elixir’s AST, unlike 3-tuples which have a special meaning. For example, the form {atom, metadata, arg} when not is_list(arg) is not valid Elixir AST! so it’s better to avoid 3-tuples and more complex expressions and working with nested 2-tuples. @OvermindDL1 has refereed to this as “escaping Elixir’s AST” and I really like the expression.

So, back to the point. Working with the nested 2-tuples is hard and the format is quite artificial, so I have combinators that make it easier to build them (and in the future maybe even pattern match on them). I’d love to use records (from the excellent Record module, which BTW should be more well known) instead, but they havd the complication that they wouldn’t compile to nested 2-tuples…

So, what’s so great about using 2-tuples exactly? It’s the fact that the intermediate representation of the compiled templates is a valid Elixir quoted expression! It’s not executable Elixir code, of course (it requires a couple transformation steps to become executable Elixir code), but is something which I can feed into Macro.prewalk/2, which makes it trivial to traverse the expression without having to manually implement a tree traversal that respects my templates’ semantics. My templates’ semantics are now the semantics of normal Elixir code.

Template widgets as macros

Last post I’ve talked about the possibility to implement reusable widgets as macros which expand into undead templates. An undead template is a value of the form:

{UndeadEngine.Segment.UndeadTemplate, {segments, meta}}

(Remember that UndeadEngine.Segment.UndeadTemplate is just an atom name like :undead_template. The advantagte of using a more verbose name like the one above is that it reduces the chance of accidental name collisions. This should be made more hygienic in the future anyway, but for now it’s good enough)

Any macro that expands into nested 2-tuples like the above can be optimized by flattening the segments and merging the static parts. As a (quite functional) proof of concept, I’ve implemented the tag/2 macro (you can find the implementation here). It works like this:

iex(2)> tag(:input, [name: "user[name]", name: "user_name", value: ""])
{UndeadEngine.Segment.UndeadTemplate,
 {[
    {UndeadEngine.Segment.Static, {"<", []}},
    {UndeadEngine.Segment.Static, {"input", []}},
    {UndeadEngine.Segment.Static, {" ", []}},
    {UndeadEngine.Segment.Static, {"name=\"user[name]\"", []}},
    {UndeadEngine.Segment.Static, {" ", []}},
    {UndeadEngine.Segment.Static, {"name=\"user_name\"", []}},
    {UndeadEngine.Segment.Static, {" ", []}},
    {UndeadEngine.Segment.Static, {"value=\"\"", []}},
    {UndeadEngine.Segment.Static, {">", []}}
  ], []}}

The argument list given to tag/2 is static, so it will generate a series of static segments. On itself, this is not very impressive.

On the other hand, this is very impressive:

iex(3)> tag(:input, [name: "user[name]", name: "user_name", value: ""]) |> Optimizer.optimize(__ENV__)
{UndeadEngine.Segment.UndeadTemplate,
 {[
    {UndeadEngine.Segment.Static,
     {"<input name=\"user[name]\" name=\"user_name\" value=\"\">", []}}
  ], []}}

The optimizer has just recognized the template as purely static, and has just merged the static parts together. Our reusable widget has been compiled into the most efficient format possible. Now let’s make it harder. Usually, an HTML widget won’t be purely static. It will contain dynamic parts. For example:

iex(4)> Macro.expand(quote(do: tag(:input, [name: "user[name]", name: "user_name", value: value])), __ENV__) |> Optimizer.optimize(__ENV__)
{UndeadEngine.Segment.UndeadTemplate,
 {[
    {UndeadEngine.Segment.Static,
     {"<input name=\"user[name]\" name=\"user_name\" value=\"", []}},
    {UndeadEngine.Segment.Dynamic,
     {{{:., [], [PhoenixUndeadView.Template.HTML, :html_escape]}, [],
       [{:value, [], Elixir}]}, []}},
    {UndeadEngine.Segment.Static, {"\">", []}}
  ], []}}

The system has detected that parts of the template are static and other parts are dynamic, and the optimizer has compiled the template into three segments: a static segment, a dynamic segment and a static segment. Again we see that the widget has been as optimized as possible.

Now let’s try it with a real template:

<% a = 2 %>
Blah blah blah

<%= tag(:input, [name: "user[name]", id: "user_name", value: @user.name]) %>

<%= a %>

Blah blah

It’s intuitively obvious that the template contains some dynamic parts and some static parts. It also contains the tag/2 macro which has an output that’s mostly static. Only the value attribute is dynamic. When we compile it, we get the following raw output (converted into Elixir code - not a quoted expression! Remember, we can do it using Macro.to_string() because the templates are valid AST):

{UndeadEngine.Segment.UndeadTemplate,
 {[
    {UndeadEngine.Segment.DynamicNoOutput, {a = 2, [line: 1]}},
    {UndeadEngine.Segment.Static, {"\nBlah blah blah\n\n", []}},
    {UndeadEngine.Segment.Dynamic,
     {tag(:input,
        name: "user[name]",
        id: "user_name",
        value: __MODULE__.fetch_assign(var!(assigns), :user).name()
      ), [line: 4]}},
    {UndeadEngine.Segment.Static, {"\n\n", []}},
    {UndeadEngine.Segment.Dynamic, {a, [line: 6]}},
    {UndeadEngine.Segment.Static, {"\n\nBlah blah\n", []}}
  ], []}}

Although the expression is quite complex, if you look carefully you can recognize the static and dynamic parts in the template above. You can also see that the tag/2 macro hasn’t been expanded yet. If we expand the macro and optimize it, we get the following:

{UndeadEngine.Segment.UndeadTemplate,
 {[
    {UndeadEngine.Segment.DynamicNoOutput, {a = 2, [line: 1]}},
    {UndeadEngine.Segment.Static,
     {"\nBlah blah blah\n\n<input name=\"user[name]\" id=\"user_name\" value=\"", []}},
    {UndeadEngine.Segment.Dynamic,
     {PhoenixUndeadView.Template.HTML.html_escape(Fixtures.fetch_assign(assigns, :user).name()),
      []}},
    {UndeadEngine.Segment.Static, {"\">\n\n", []}},
    {UndeadEngine.Segment.Dynamic, {a, [line: 6]}},
    {UndeadEngine.Segment.Static, {"\n\nBlah blah\n", []}}
  ], []}}

As you can see, the static parts of the tag at the beginning and at the end have been merged into the static parts before and after the tag, so minimize the number of segments.

As long as the macros expand into the appropriate format, thee optimizations are always possible.

What’s missing

I’ve taken a big detour with the goal of being able to expand macros inside the templates and optimize their result. I now have a very general framework to create optimized widgets, and it’s easy for other users to create their own libraries of optimized widgets using some basic combinators.

What is now missing is a way of compiling these templates into Elixir code that actually generates the iolist with the rendered template. The implementation doesn’t pose any particular challenges, and I’ll get to it soon. As you probably remember from prior posts, older versions of this project already had the capability to compile into quite efficient iolists, it’s just that the internal format of the templates has changed so much that I’ve had to scrap the old compiler and must reimplement a new one.

The basic idea is simple: you just need to compile the segments tagged as UndeadEngine.Segment.UndeaedTemplate into blocks that run their expressions in order, assign them to variables and return them as a list. It will be even simpler than what I was doing in previous versions because before I was actually compiling the templates into Elixir AST and then parsing that AST again, just to finally compile it into AST… I can be much more efficient now.

Source code

The code is, as always, on Github: GitHub - tmbb/phoenix_undead_view: EEx engines that compile Phoenix templates into static and dynamic parts for better diffing over the network · GitHub

It’s still a little rough and some parts need to be refactored. The meat of the project is in the Tag module, which implements the “self-optimizing” tag/2 macro and the Optimizer module, which expands the macros in the template and optimizes it as much as possible by flattening it and merging the static segments together.

EDIT: Even if it turns out this architecture is not a good fit for something like LiveView, I have shown that with the thelp of macros such as tag/2 and with the optimization steps I can produce much better code than the default Phoenix templates and their functions-based widgets which must rerender everything (even some parts tat are actually static) each time the template is called. As I’ve said before, I believe the Phoenix engine could take some ideas from this implementation.

This series of posts is very interesting from a “historical” perspective on the Vampyre project. Even I like to re-read them sometimes, but I think I should write something more definitive, like a blog post not that things are stabilizing somehow. Even for people who are completely uninterested in rendering HTML templates can learn some interesting things from it. The Vampyre project touches some interesting areas, such as:

• How phoenix actually renders templates
• Tricks to get performance on the BEAM (iolists are fast, but not that fast; also, function calls come with a cost)
• Using macros to avoid doing at runtime what you can do at compile-time
• Manipulating Elixir quoted expressions
• Working with EEx engines (the documentation we have now is somewhat lacking; the hexdocs could use some love)
• The importance of a smart intermediate representation in a compiler (don’t try to output the final code in a single step)
• Using macros to delay expression evaluation
• Inlining functions without explicit compiler support

I think I should write something in the line of the following (amazing!) blog post: https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2-io-lists-in-phoenix/. Which by the way was the major source of inspiration for the whole Vampyre project. After all, the big idea for this came from that post: Phoenix automatically and universally applies this simple view caching strategy: the static parts of our template are always cached. The dynamic parts are never cached. The cache is invalidated if the template file changes. The end. The Vampyre project just takes that idea into the next level, by aggressively merging the static parts of the template.