Or are you just talking about reimplementing TEA on the server? If TEA is so fast I should youse it to handle the diffs on the client, right?
And more importantly, can I render the initial
HTML using vampyre/phoenix/whatever (using Elixir) and then use the VDOM implementation from TEA to mutate the DOM using a vdom fragment sent by the server?
The attraction of using morphodom is that it patches simple HTML strings into the DOM with no further intervention…
Sorry for spamming my own thread, but if you don’t like the incoherent stream-of-consciousness style, then you have likely stopped reading weeks ago…
I’ve been thinking about @OvermindDL1’s suggestions and something’s not right… There are two phylosophical positions:
-
Templates should generate valid (X)HTML which is easy to analyze statically. The poor user is weak of spiritand cannot be trusted to generated valid (X)HTML unaided. Also, HTML5 is a freak show and should be erradicated from the face of the earth, not to taint it with its basphemy. This way, you can easily diff and optimize templates in a semantically meaningful way.
-
Templates should generate whatever binary the user feeds into it. Because the use knows best, the template engine has no type safety whatsoever (Types? What types? The only relevant type for web development is the byte and the iolist!). The template engine better generate exactly what the user asks for! Which will be absolutely valid HTML5 (by pure coincidence, it could be a JPEG file or something like that). This allows things like these:
<<%= tag_name %> <%= attr_key_and_val %>>(Closing tag? do I need a closing tag? is this the 00’s?). Semantically, that’s a very dangerous template fragment. It assumes the variables will be strings/iolists that make sense in the context of the rest of the template. And it’s impossible to analyze statically…
So, I’m working firmly in option number 2, while @OvermindDL1 wants to push me towards option number 1. @dgmcguire is working firmly in option 1. Clearly, both extremes (1 and 2) have merit. Option one holds the key to optimizations both on the server and client. Option 2, well, is compatible with the way Phoenix renders templates and has given me a 12x performance improvement. Which is something…
Is there any merit in an attempt o reconcile both options? Clearly option 1 could benefit from the performance of Vampyre for serverside rendering. But to send the actual minimal diffs, a format which is easy to analyze statically is very important. The VusJS web framework has string templates, but those templates have many restrictions. For example, tag names need to be literals (not variables). IIRC attribute names (not keys!) need to be literals too. In my option, those options make sense… But if there is such thing as a Phoenix way, I think they go against it. The Phoenix user expects to be able to send whatever they want to the browser. They want the freedom of string-based templates. My approach gives us that. As does Chris’ in Phoenix.LiveView.
You generate whatever binary you want, and let the browser make sense of it on the other side. The only difference is that no the browser needs the morphdom javscript library to make sense of what you’re sending. This is very powerful model (even if a little inefficient on the client), and I’m having a hard time letting go of it. On the other hand, I do see the beauty in a system that enforces valid (X)HTML by default…
So, my conclusion is that both extremes (1 and 2) are valid, while the intermediate solutions look like a big mess. Not that one can’t successfully navigate the messy parts, but it’s not going to be easy.
I need to decide whether to commit to option 2 or try an intermediate approach in which everything must compile to a rigid structure that respects option 1. Why is this important? Because if I want to follow option 2, I must reimplement all widgets in Phoenix.HTML as self-optimizing macros. Which is a lot of work. If I decide to implement restrictions that put me firmly in option 1, then I can spare most of that work, because things are much simpler to optimize.
2 Likes
I hooked it into an existing benchmark that tested a lot of other things like Elm, Vue.js, React, Angular2/3, etc… It was consistently at least on par with Elm or faster, with nothing else being close. 
Just make your own patcher based on whatever format you encode it in, tea doesn’t seem appropriate for this (although you ‘could’, it would very much want to ‘own’ the DOM and not work with any third-party things, where something like Drab does). And that benchmarking was patching into a real DOM.
I’m talking about not using that style at all, that would imply complete ownership and requires more iterating of the DOM than is entirely necessary since you can pre-bake known information at compile-time, something like a JS client that owns its part of the DOM can’t really assume that easily.
That still involves iterating the DOM, there’s no point of that, not even Drab iterates the DOM, it literally uses the element ID’s to acquire the element that it directly needs to modify, significantly faster, and if you bake in the known ID’s in the template and into specialized javascript code for each of those elements then it becomes even faster still.
Lol, I enjoy it, similar to how my mind works. ^.^;
When you are dealing with javascript and all such, HTML5 should be used and enforced, it is the current web standard and will be for the long forseeable future.
This should not be assumed since the whole concept of patching via javascript and all falls flat with it.
I personally don’t see option 2 having merit personally, you are throwing out the known domain that this thing will only work in, it won’t work in any other so making it generic outside of the only domain it will work in seems to be losing capability for no reason whatsoever?
Option 2 is perfectly sufficient for non-HTML templating, and it should become the standard EEX engine style, but for being able to peel apart element and hotpatching on the fly a remote client canvas then specific knowledge is far superior. This is going to require 2 implementations of course though, but a lot of shared code between them regardless.
I’d personally see it being 1 library with 2 engines (maybe more if you start supporting hot-CSS patching or whatever) with a shared common set of code but specialty implementations where the, for example, HTML5 version spits out not only the bindings and template on each return (like the generic) but also a listing of updateable ID keys and mappings and so forth.
Plus with knowledge of the problem domain then having something like Phoenix.HTML’s tag could literally return some specialized HTML5 tuple object or something (for easy compile-time processing), being significantly easier to make.
1 Like
If you have a sensible representation for HTML (i.e. 3-tuples or something like that), then yea, it is trivial. And I can certainly do it. I even have a number of strategies to parse an EEx template into HTML at compile-time very efficiently.
You do get that this is the whole point of using morphdom, right? Morphdom takes up a string representing HTML and patches it into the DOM.
And as I said above, this is the “Phoenix way”. Phoenix templates are things that compile into arbitrary iolists with no “type safety” whatsoever. Contrast that with a haskell web framework like Yesod, which enforces valid HTML at compile time with a mixture of macros and the type system. I would like to work with templates that can be proved to be valid (within reason, of course, no need to go full Applicative-Monad-Functor-Endomorphism + dependent types or whatever is in vogue nowadays). But I’m working within Elixir (and Phoenix!), and trying to validate things at compile time would go against the spirit of both.
Currently, I can just tell someone who wants faster templates to replace use Phoenix.HTML by use Vampyre.HTML. If I start to require type safety and removing flexibility when writing templates then I’ll be creating my own little type-safe world incompatible with everything else, which will be used by a total of 3 people, including both of us 
By working within the conventions of the Phoenix ecosystem (even if I don’t necessarily agree with them), I make the benefits of Vampyre templates available to the wider community.
I think the merit of option 2 is explained above. Again, option 2 is not something I like philosophically, but it’s certainly pragmatic, given the constraints of the Phoenix ecosystem.
Option 2 is more or less the way non-HTML templating works. What I don’t think you are appreciating is that what makes option 2 fast in Vampyre is not only the template compiler itself, but also the library of macros for HTML templating. Option 2 is fast precisely because I’m replacing HTML widgets previously implemented as functions (AKA things that do work at runtime) by widgets implemented as macros (AKA things that do work at compile-time)
My Vampyre.HTML.Tag.tag macro already returns a specialized structure for use with Vampyre templates, namely a Segment.container(), which contains a list of static and dynamic segments. Unlike the default Phoenix.HTML.Tag.tag function, which builds an iolist at compile-time. Vampyre’s specialized structures make it probably the fastest possible template implementation for EEx (without messing with the elixir compiler)
I’m perfectly aware of how much simpler my life would be if I implemented the tag widget as you say… In fact, I could probably implement something like Texas in about a week (with proper compile-time optimizations) with the experience I have now, but I’ve deliberately decided to work with binaries alone and leave the hard work of rebuilding/iterating the DOM to morphdom (as @chrismccord has done, BTW).
But in the aggregate you’re obviously right. I’ve built a suboptimal solution out of laziness alone. The correct thing to do would be to team up with @dgmcguire and work on a proper VDOM implementation on the server, as you suggest. It would have a number of advantages:
-
It would be as fast on the initial render as Vampyre is, because I can always compile HTML into super-fast iolists using Vampyre’s optimizing compiler.
-
It would be faster to diff than anything that uses morphdom (I’m taking it on faith because you’re saying it and it’s intuitively obvious, and I won’t become a javascript pro just to benchmark both options…)
-
I can trade higher CPU usage for better diffs pretty transparently (or not, if I decide I want to spend CPU cycles on the client instead of on the server). I’m thinking concretely of diffing lists on the server to send smarter diffs like “replace 3rd child of element with id
128736182”, instead of sending the full child list and leaving it up for the client to fix things. -
Having a VDOM on the server is certainly good from a marketing perspective. After all, everyone and their mother know (or think they know) what a VDOM is, while saying “it’s the fastest iolist renderer around” just makes people say “WTH is an iolist?”
For simple cases (setting attributes and stuff like that), I can do it manually. For more complex cases, like replacing the children of an element, I can always use morphdom. It’s a pretty good patcher actually. Or I can reuse some other VDOM library and sending pre-made patches from the server (encoded as JSON, BERT, HTML or whatever)…
1 Like
Just don’t count on Floki to get the parsing right if you want HTML5.
Don’t worry, I don’t count on anything at all to parse HTML5 right
Although Floki actually supports self-closing tags, which is actually pretty good:
iex> Floki.parse(~s'<input value="x">')
{"input", [{"value", "x"}], []}
That sounds fairly horrifyingly inefficient… ^.^;
Depending on how it works (swapping?) it could cause lose of HTML state too (like editing an input box that suddenly clears just because something was patched).
Oh that sounds blissful. ^.^
But still, Floki can verify and parse it quite well.
For an HTML parser sure, Floki would verify it is all written properly (or the user would get warnings/errors about invalid html, I’d opt for errors as that would be a bug regardless). For the generic non-html parser, that’s stimple string munging then.
That also means that people that want to make their own things have a lot more hoops to jump through because it is designed to be implanted into a generic system instead of something domain-specific (as HTML is).
Eh, don’t need a VDOM on the server, only at compile-time where it’s information is used to build the specific processes to perform efficient hotpatching, the vdom would not exist past compile-time.
Yep, initial render would be virtually indistinguishable to a generic templater (aside that it should return and/or inject javascript for fast in-position hotpatching).
A quick look at morphdom makes it look like it is passing the string to the browser and having it create all elements, then it transforms the differences from the instanced elements to the DOM elements, then destroys the instanced elements… that seem pretty horrifying… ^.^;
Some quick googling on performance of it shows it is not too bad at element to element diffing (understandable, they are native), but the instancing of the string to an element tree is not very fast, and there are hicks in FPS and performance when the elements are later garbage collected. It is not something you want to call on large DOM trees or with rapid changes (both of which Drab entirely and fully supports without issue).
Yep this here! I think there is even a JSONPatch spec that would make that more simple too.
I thought it added an HTML5 parser not too long ago that was optionally selectable? It’s mainly useful if you intend to ‘edit’ the html anyway, if just reading the meeseeks is better/faster.
I havn’t seen Floki go wrong with any of my HTML that I’m using with Drab for near on a year now though. Do you have examples of correct HTML where it fails?
I’s a parser written in Rust for Firefox. You definitely don’t want to require an instalation of Rust in the user’s machine to compile templates.
It’s only needed at compile-time, not runtime, I deploy to a rust-less docker alpine container without issue.
It does.
It doesn’t support this by default, but you can hook some callbacks to avoid this problem. I think…
You and the 4 other people here who know what OCaml is xD But yeah, that’s the tradeoff for strong typing: you reject some sound programs (yes, it’s possible that the totally untyped string the user passed into the template is valid HTML!) in exchange for better guarantees.
The main issue here is that it’s not that obvious how to support tags with attributes with dynamic keys. But yeah, it’s a restriction few people will hit in practice.
People using Floki generally go with the default parser which is the Mochiweb one- particularly people using Floki as part of a library, so the library’s users don’t need to have Rust deps. I’m not sure Floki’s html5ever parser is even working on the latest OTP?
Floki (with the mochiweb parser) generally fails with incorrect HTML, or rather, in how it parses HTML that is “valid” HTML5 but may get moved around.
Edit: Simple example:
<html><head></head><title>Hello, TEA</title><body></body></html>
Parsed by Floki with the mochiweb parser to:
{"html", [],
[{"head", [], []}, {"title", [], ["Hello, TEA"]}, {"body", [], []}]}
Parsed by Floki with the html5ever parser to:
{"html", [],
[{"head", [], [{"title", [], ["Hello, TEA"]}]}, {"body", [], []}]}
So let me guess, you’d “Build In” some known ones like textbox, and leave custom elements up to the user to implement via some other plug that they’d have to do or so? That sounds remarkeable painful when it’s otherwise simple attribute/property changes on them… o.O
Speaking of, I’m not seeing an immediate way to distinguish between attribute or property updates in that library?
If it’s using the firefox HTML5 parser, I don’t see how that would happen though. ^.^
Actually that’s a super common use-case at work. We use an excessive amount of custom elements along with is="..." enhancements to have full support for javascriptless things like old screen readers we have to deal with to newer things to evergreen, it works fantastically.
I know html5ever is working on the OTP 21.1.1 as I’m using it.
Do you have an example though, I’ve not run in to any such issues in Drab as of yet. (NinjaEdit: It looks like Drab is set to use floki with html5ever on my server, so that might be why I don’t see issues?)
Edited above to include, but here’s simple example.
<html><head></head><title>Hello, TEA</title><body></body></html>
Parsed by Floki with the mochiweb parser to:
{"html", [],
[{"head", [], []}, {"title", [], ["Hello, TEA"]}, {"body", [], []}]}
Parsed by Floki with the html5ever parser to:
{"html", [],
[{"head", [], [{"title", [], ["Hello, TEA"]}]}, {"body", [], []}]}
Ok, but that makes some optimizations difficult. Although is the goal is to render things intos a JSONPatch I guess that’s not a big problem, as I’d have to have a map to send to the JSON encoder anyway. But it’s a problem with my string-based templates (I can render it dynamically, I just can’t optimize it).
I think you’re the only human in the whole world for whom that distinction matters 
I’ve learned it from you, and aside from your bucklescript-tea repo and the js library that inspired it (https://github.com/Matt-Esch/virtual-dom), I haven’t seen that distinction anywhere. So, no, it has no ways to distinguish it. I guess that if I must implement my own VDOM I need to understand the distinction, right?
Actually it is extremely common as changing the wrong one on certain elements will not work (or in some notable cases throw an exception). Everything from my tea library to vue.js to polymer to react to near everything I’ve ever seen has the distinction. Most will often build in known mappings so you don’t have to deal with it on things like className and so forth, but they all have ways to override it as well since custom elements are of course custom, thus only the author of the site can known for certain hence why a way to specify which to use must always be necessary (and yes, Drab has a way to specify as well).
Regarding the part where input components might be overwritten, how do you propose to solve that without special logic on the client? By customizing the patches on the server so that they never edit the value of input tags? Isn’t that the same as the whitelist approach you don’t like?
Do you have a link please?
