Good advice, thanks
1 Like
Why do you need an ETS table here? Don’t you plan on having a process per client? You already require a process per client if you’re using a phoenix Channel. A Phoenix channel is pretty much a GenServer (maybe it’s even an actualy GenServer?). Why don’t you store the state of the client in the process itself? What do you gain fron an ETS table?
1 Like
You gain being able to share a cache between all clients that are up-to-date and viewing the same page. Reduces the memory footprint significantly if you have many clients
1 Like
Oh, I get it. You want to share live data between many clients. That will work, yeah. But for my primary use cases (quick feedback with forms) it wouldn’t help much.
1 Like
Actually, I think I could write an EEx engine that does more or less that. It’s not too hard. There are two main ways of doing it
-
You either parse the HTML text with floki and then use an EEx engine to compile the dynamic parts into quoted expressions or
-
You parse the template into a list of segments, you replace the dynamic segments by something inoccuous (like a unique string, UUIDs, etc), run Floki on that and then replace the UUIDs by the quoted expressions.
I don’t know, the question is that as long as you have a smart intermediate representation, compiling stuff into that representation should be very easy.
3 Likes
The best resource on how to implement a VDOM is probably this: https://medium.com/@deathmood/how-to-write-your-own-virtual-dom-ee74acc13060
It’s in Javascript but it’s very easy to follow. The main problem is that actually translating that code to Elixir is quite boring, but once you do you have a working implementation of a VDOM. It might need some tuning, but you’ll have the basics.
WHEN you have that, you might think about isolating static from dynamic parts of the trees, building a dependency graph, etc. To get diffs from maps you might want to look at this: MapDiff: A small library that computes the difference between two maps
I don’t know how many people are using it in practice, but it seems pretty solid. As long as your data is in the form of maps and lists it should be useful.
EDIT: just make sure that diffing the data is actually faster than building the VDOM and diffing that.
2 Likes
If you want to convert your 3-tuples (or any intermediate representation you pick) into an IOlist for initial rendering, you can use Vampyre’s optimizing compiler. It’s even easier to use if you start with something like 3-tuples (instead of starting with EEx templates and special macros).
3 Likes
@dgmcguire, just one final piece of advice. As I said above, it’s good if your intermediate representation is valid Elixir AST. This means that representing HTML with 3-tuples is a bad idea. You should do like me and use nested 2-tuples, which are valid Elixir AST. That way you can expand macros with Macro.prewalk/2 instead of implementing your own function to recursively walk through the AST.
Instead having something like {tag, attrs, contents}, you should have something like: {:html_element, {tag, {attrs, contents}}}. Because this is hard to type, you should have a macro construtor such as h/3 so that you can write h(tag, attrs, contents) instead. Using a macro is better than a function because you can pattern match on the output of macros, and you can’t do so on functions.
I’ve mentioned this trick before, but this is just a reminder that it applies to your case too (it applies anywhere you want to expand macros inside a complicated structure).
3 Likes
This is super important. You don’t necessarily need to tag ‘everything’, but at least at key points where ‘bulk’ changes tend to happen to immediate or near immediate children. My OCaml lib is a simple O(N) Diff walker that walks the intersection of the old and new diff’s along with the DOM tree all simultaneously to apply differences with lots of opportunities for early-outs and so forth. Not bad when you don’t have or want observeables.
In something like this project you could easily just ‘tag’ the dynamic parts (or a parent there-of in the DOM) and send just the changed “data” itself to the client to re-apply to the proper areas based on the ID links to that specific data. Would be hard to be faster than that while remaining as tiny as possible.
You should diff on the client. What the server should do is just uniquely identify immediate or parent nodes for each dynamic data writing along with a list of such links that would be written out to the javascript of the page, then you’d just need to send the dynamic data changes to the client on change, no need to iterate a virtual DOM or anything, and let the client directly access what it needs to change for each named dynamic data instance that was changed. Super fast then and really minimal CPU cycles on both server and client both.
This is the observable pattern for note, just encoding it into the page directly via information generated at compile-time in the template. No diffing, just ‘links’ between named dynamic data and where the updates should occur that the client will know about and apply efficiently.
2 Likes
So you’d build “DOM” nodes which are just raw strings that represent an HTML element with a given ID, and send a message like {:replace, id, raw_string} to the browser? That way the browser would know which element to diff… Interesting.
The problem is that it’s far from a minimal diff. Suppose you insert a single element into a list. You’d want something like: {:insert, index, raw_string}. But this requires diffing the tree (i.e. walking at least a level of the VDOM.
Where id is not the element ID to update, but is rather the unique name of the dynamic data to update, then the client would have a mapping of the dynamic data names to many DOM ID’s (many being however many times it was used in the template) to then update, baked into the page.
Nope, no VDOM at all, no walking. The javascript will have to be ‘smarter’ to know the semantics of how to insert some code, but offloading that work to the client is more scaleable for the server and can significantly reduce network load, especially when combined with piecemeal JSON updates of the passed over data itself.
1 Like
That’s what Vampyre already does! It only goes one level deep, though.
Exactly! And one level is fine as long as it’s all flattened, that is “Optimal” actually. Now you just need to autogenerate some unique ID’s to put in the DOM, and of course parse the DOM itself (Floki or so) at compile-time to be able to do that, and pass the requisite rebuilding information into generated javascript to be injected into the template, and done. 
1 Like
Look at the first post in this thread. Is that that you mean?
1 Like
Similar idea but more optimized yeah.
But how is yours more optimized? Because you only update a certain ID instead of rebuilding the whole DOM
1 Like
That is a big big one there but the dynamic data ID to DOM ID-list mapping also allows for extremely efficient in-place updates in the DOM without needing to construct elements at all unless necessary (creating new one for example of course). Mutating the DOM is surprisingly cheap, rebuilding the DOM is lag-hell.
That’s not what the morphdom guys say 
But I believe you’re right… I should do what you say, but that requires doing pretty much what @dgmcguire wants to do. But Vampyre templates are still useful in these circumstances. They are very fast at rendering the HTML to strings.
It’s like I’ve told @dgmcguire: this requires an intermediate representation which I can compile to a highly optimized string representation while I simultaneously keep enough semantic information to add unique IDs for everything.
Writing an EEx engine that compiles into valid HTML poses some challenges, though. It would require (seemingly) arbitrary restrictionson what you can put in a template. Drab already does more or less that. I don’t know if the high complexity and unintuitive interface is worth it.
EDIT: The problem is that users expect to be able fo use normal EEx templates with no restrictions. That is possible with vampyre, but some restrictions are needed for something that compiles to a server-side DOM.
I’ve tested quite a bit and that’s how it always came out. ^.^;
Why do you think bucklescript-tea is so fast (it blows react away as just one example, faster than vuejs and all)? 
Observeable updating is even faster as it’s the same but without the iteration.
Ok, I take it on faith that bucklescript-tea (just TEA from here on) is wicked fast. Bur how do I capitalize on that? TEA seems to be diffing a custom stripped-down VDOM. Is it still fast when it comes doen to patch it ino the real DOM?
How could I feed TEA the DOM fragments from the server in an efficient way? As JSON?
Should I render a webcomponent using TEA that is driven by messages from the server?
