Well, it goes both ways xD If I had started thinking about how to optimize for comprehensions (instead of quitting), I’d have stumbled upon your solution about 1 month ago It’s a natural consequence of trying to do things at compile time, because we can take advantage of the fact that for is a special form that can’t be overriden, so it can be analyzed statically.
But I believe we may have started with different priorities.
You had @chrismccord’s example from the LiveView talk and and maybe optimizing such dynamic highly dynamic templates was a priority for you.
I started thinking about (mostly static) forms and how to optimize those, because those are the ones I saw myself using the most, especially the part of having real-time form validation without writing any JS. That brought me naturally close to the idea of inlining static parts and merging adjacent binaries together. Yes, live tables that could update in response to typing in a search field were cool, but I had already given up on those
I’m the first to praise the simplicity of your implementation but I think most of the complexity with my approach is unavoidable. It’s the result of implementing an optimizing compiler (although a very simple one) that tries to separate the static and dynamic parts in a very obsessive way.
The other main source of complexity with my approach is probably the attempt to maintain backward compatibility with Phoenix.HTML, which contains some things that are not very easy to optimize statically. The tag() function, in particular, is a bit weird, and I should probably break compatibility so that it can be abetter building block for the rest. If I can encapsulate most of the complexity inside the tag() macro (because I need it to be a macro), thins will probably be much simpler in the rest of the code base.
I don’t consider this “premature” optimization, because it’s essential for my main goal, which is to send the absolute minimum amount of data over the network for things like forms.
Weren’t you looking at compile time all along? So you should have seen the optimization.
Seriously though, that’s only part of the solution. The fact the comprehension is communicated via a struct, similar to the Rendered struct, arose from the fact we are also tackling runtime. The current implementation can easily nest templates (<%= render ... %>) without traversing or nesting ASTs and we could extend that to comprehensions too. So we are mixing both approaches instead of one or the other.
I am focusing rather on what comes generated on phx.gen.html, and you see both nested templates and comprehensions in there.
It is unavoidable only if you assume it HAS to be done. Most likely it doesn’t and there are probably other ways we can optimize things like forms, looking at its domain in particular and not necessarily at the template engine as a whole. Edit: But of course, doing Vampyre is a great exercise in itself.
I agree, that’s what makes me feel so stupid about this I’ve stumbled upon the idea of flattening the highly nested EEx templates as a way to separate the static and dynamic parts, had all this work reimplementing Phoenix.HTML as macros (as you can see by line count of the Vampyre project this was not a trivial effort), built an optimizing compiler around the idea of merging the static parts and then missed a really obvious optimization which you thought of on day 2 of working on this So yeah, feeling pretty stupid right now.
Yes, the moment you mentioned your comprehensions optimization I noticed it was a natural consequence of your work with template fingerprinting, and I assume you thout of that optimization because of that. The fingerprinting idea can be taken pretty far, namely optimizing different branches of a case statement. The <%= render ... %> thing is not very important to me, because I believe I can have a <%= vrender ... %> macro which dumps the inner template inside the outer one in a way I that it can be inlined, but it’s a great idea anyway.
Yes, I know it is. There are still things from Vampyre which I think you can implement for further performance gains. The main one is something I’ve stolen from Drab (from which I plan on stealing more stuff, of course).
And I like the conceptual elegance of sending only the absolute minimum of dynamic data, so I’m likely to continue working on Vampyre for some time.
But since you’re thinking about breaking compatibility in the form_for/4 function, I will now break compatibility with the tag() function (in my case, the tag macro) so that it works in a more semantic way. Hopefully, I’ll be able to reuse my reimplementation of the tab macro if I ever want to go the way of a VDOM (Vampyre DOM, of course ), as @OvermindDL1 is always saying I should do (and more or less like Drab already does in its own way).
We wouldn’t break compatibility, rather we would introduce new arities. Which changes you had in mind for tag?
Yes, exactly. It is a possibility… but I am not sure if we will leverage it there. That’s also another way we could optimize form_for: form_for would receive a fn that returns a Rendered struct and the form_for could attach new static and dynamic information to the Rendered struct and return it.
Any of them do not rely on macros? If so, I would love to hear them.
There is also the other idea which is to introduce something like <%| |%> as optimization case of something you know that won’t change, even if they are dynamic (such as labels, inputs, etc). But I would wait a bit before introducing user fine-grained control.
Yeah, you’re right. That doesn’t break backward compatibility.
I’d like it if tag worked like content_tag when given a list of arguments. Something like:
tag(name, attrs, children)
That would make templates more “semantically correct”, I think. That would be the way forward if I wanted to transition Vampyre’s inner workings into a VDOM. Currently, the tag/2 function incentivizes template authors to dump unclosed opening tags (like you’re doing with form). But it’s a pretty minor issue, and maybe not worth breaking backward compatibility over…
You probably mean something like <%| ... %>, because the thing you’ve written is not valid EEx.
FWIW, LiveEEx assumes that any code without assigns is never reloaded. So we kinda get the <%| %> behaviour for “free”. We will force devs to push their changes through assigns.
Also, huge thanks to @grych for braving a lot of this more than a year ago and forcing us to improve the template engine. One of the reasons we can do this now so trivially is because of handle_begin and custom terminators and other stuff that we wouldn’t have added to EEx without him pushing EEx to its limits at the time.
Sorry, I was still thinking in “Vampyre”… In your architecture that doesn’t make as much sense. I can see it being used in the name for input tags: <input name="<%| form.name %>[field1]" ...>.
The [fieldI1] suffix is already purely static in Vampyre, as long as you supply a compile-time constant (like the :field1 atom). I’m not declaring the form.name as “fixed” in Vampyre yet because it seems a little too risky: what if the user wants to replace that with a form with a different name? But I think declaring that as fixed (i.e., something that doesn’t change after the initial render) is the right choice.
Currently in Vampyre, I have 4 kinds of segments:
static, which never change (normal template text)
dynamic, which change each time the assigns change (<%= ... %>)
support, which are executed but aren’t rendered (<% ... %>)
fixed, which are rendered once and then never change (<%/ ... %>); this is good for CSRF tokens and static translations (which never change after the initial page render)
container (<%= content %>, where content is a macro that is expanded into a certain format). Container segments can be flattened into the outer template and have their static bits merged.
I think that adding the fixed type to LiveEEx is worth it because of some translations that will never change. In a completely localized page, maybe most of the text will be translations. But if those translations don’t depend on the assigns, that’s not a problem to you, I guess.
On further thought, if your “change” to form_for maintains compatibility, I guess i’ll maintain it too. That way, Vampyre will continue to be a drop-in replacement for Phoenix.HTML and can remain available for the total speed junkies who want to try it without changing their templates (assuming I can keep up with al your awesome oprimizations, of course!)
IMO, @grych did pretty much everything that LiveView claims it will do, just with different tradeoffs which naturally led to a different design in the templates.
The main difference is that LiveView (and Vampyre, which derives from the proto-vaporware LiveView demo) are letting the browser handle the entirety of the DOM with the morphdom library.
If one decides not to use the morphdom library (which offloads complexity to the client), them onr mist implement something like Drab. It’s possible that Drab could be enhanced with morphdom with interesting effects too.
The main problem with something like Drab, which tries to preserve the semantic meaning of the HTML is that there are some restrictions on where you can put some HTML tags. Id you had a “neutral” HTML tag which you could nest literally everywhere (say, a <template> tag), then Drab would be clearly superior to LiveView and Vampyre (and a partial server-side VDOM solution might have been even better).
Basically, what LiveView and Vampyre are doing is just a reinvention of Drab in the morphdom world. It’s not clear that our aproach will lead to better performance or a better API than Drab.
I should write about some of the ideas I have fot the API of live controllers.
Right, we are all working on the same domain and we are all exploring different trade-offs at the template level. Texas also tries a different approach, correct?
But I would say those trade-offs came from doing different choices at the high level, user facing API, rather than the opposite (i.e. the template is not dictating how the user API should work).
Yes, that’s true, but it’s also true that LiveView’s design is only made possible because you’re willing to use morphdom on the client. Fortunately for us, morphdom allows us to oretend that the DOM doesn’t exist on the server and simply send binary segments.
Which is a very powerful idea, of course, but one which costs you some performance on the client.
A big difference between LiveView and Drab on one side and Vampyre on the other the first two force the user to manually update the assigns one by one, while I’d like to alllow the user to pass a completely new group of assigns with little performance impact. So yeah, there is a difference in API here that’s impacting the design choices.
A little offtopic: I’ve had trouble falling asleep last night.
I’ve spent a lot of time thinking about keeping the HTML generation as semantically correct as possible.
That would allow me to send VDOM (again, “Vampyre DOM”, not “Virtual DOM”) fragments to the client (encoded as json, for example, which is even more compact than raw HTML for most use cases). That would allow me to build a Marko VDOM on the client very cheaply and diff that against the Real DOM using morphdom. This would bypass the slowest part of Morphdom, which is to build a DOM from parsing raw HTML strings.
The main problem with this is that I’d be moving more complexity towards the server in exchange for a more performant client. And I have no idea on how to run meaningful JS benchmarks lol. This wouldn’t kill performance on the server, because most of the complexity would happen at compile time. I hope.
LV allows assign(socket, a: 1, b: 2, c: 3) and it will will send the minimal dynamic segments as necessary in a single payload, so I think we are there?
Yes, you are there. Vampyre is not there by design. I’ve made the conscious choice of avoiding that optimization for the time being. Sorry if I wasn’t clear.
EDIT: this is a criticism of Vampyre, not a criticism of LiveView
It’s a natural consequence of trying to do things at compile time, because we can take advantage of the fact that 
), as 
