Why do you need an ETS table here? Don’t you plan on having a process per client? You already require a process per client if you’re using a phoenix Channel. A Phoenix channel is pretty much a GenServer (maybe it’s even an actualy GenServer?). Why don’t you store the state of the client in the process itself? What do you gain fron an ETS table?
tmbb
dgmcguire
You gain being able to share a cache between all clients that are up-to-date and viewing the same page. Reduces the memory footprint significantly if you have many clients
1
tmbb
Oh, I get it. You want to share live data between many clients. That will work, yeah. But for my primary use cases (quick feedback with forms) it wouldn’t help much.
1
tmbb
Actually, I think I could write an EEx engine that does more or less that. It’s not too hard. There are two main ways of doing it
-
You either parse the HTML text with floki and then use an EEx engine to compile the dynamic parts into quoted expressions or
-
You parse the template into a list of segments, you replace the dynamic segments by something inoccuous (like a unique string, UUIDs, etc), run Floki on that and then replace the UUIDs by the quoted expressions.
I don’t know, the question is that as long as you have a smart intermediate representation, compiling stuff into that representation should be very easy.
3
tmbb
The best resource on how to implement a VDOM is probably this: https://medium.com/@deathmood/how-to-write-your-own-virtual-dom-ee74acc13060
It’s in Javascript but it’s very easy to follow. The main problem is that actually translating that code to Elixir is quite boring, but once you do you have a working implementation of a VDOM. It might need some tuning, but you’ll have the basics.
WHEN you have that, you might think about isolating static from dynamic parts of the trees, building a dependency graph, etc. To get diffs from maps you might want to look at this: MapDiff: A small library that computes the difference between two maps
I don’t know how many people are using it in practice, but it seems pretty solid. As long as your data is in the form of maps and lists it should be useful.
EDIT: just make sure that diffing the data is actually faster than building the VDOM and diffing that.
2
tmbb
If you want to convert your 3-tuples (or any intermediate representation you pick) into an IOlist for initial rendering, you can use Vampyre’s optimizing compiler. It’s even easier to use if you start with something like 3-tuples (instead of starting with EEx templates and special macros).
3
tmbb
@dgmcguire, just one final piece of advice. As I said above, it’s good if your intermediate representation is valid Elixir AST. This means that representing HTML with 3-tuples is a bad idea. You should do like me and use nested 2-tuples, which are valid Elixir AST. That way you can expand macros with Macro.prewalk/2 instead of implementing your own function to recursively walk through the AST.
Instead having something like {tag, attrs, contents}, you should have something like: {:html_element, {tag, {attrs, contents}}}. Because this is hard to type, you should have a macro construtor such as h/3 so that you can write h(tag, attrs, contents) instead. Using a macro is better than a function because you can pattern match on the output of macros, and you can’t do so on functions.
I’ve mentioned this trick before, but this is just a reminder that it applies to your case too (it applies anywhere you want to expand macros inside a complicated structure).
3
OvermindDL1
This is super important. You don’t necessarily need to tag ‘everything’, but at least at key points where ‘bulk’ changes tend to happen to immediate or near immediate children. My OCaml lib is a simple O(N) Diff walker that walks the intersection of the old and new diff’s along with the DOM tree all simultaneously to apply differences with lots of opportunities for early-outs and so forth. Not bad when you don’t have or want observeables.
In something like this project you could easily just ‘tag’ the dynamic parts (or a parent there-of in the DOM) and send just the changed “data” itself to the client to re-apply to the proper areas based on the ID links to that specific data. Would be hard to be faster than that while remaining as tiny as possible.
You should diff on the client. What the server should do is just uniquely identify immediate or parent nodes for each dynamic data writing along with a list of such links that would be written out to the javascript of the page, then you’d just need to send the dynamic data changes to the client on change, no need to iterate a virtual DOM or anything, and let the client directly access what it needs to change for each named dynamic data instance that was changed. Super fast then and really minimal CPU cycles on both server and client both.
This is the observable pattern for note, just encoding it into the page directly via information generated at compile-time in the template. No diffing, just ‘links’ between named dynamic data and where the updates should occur that the client will know about and apply efficiently.
2
tmbb
So you’d build “DOM” nodes which are just raw strings that represent an HTML element with a given ID, and send a message like {:replace, id, raw_string} to the browser? That way the browser would know which element to diff… Interesting.
The problem is that it’s far from a minimal diff. Suppose you insert a single element into a list. You’d want something like: {:insert, index, raw_string}. But this requires diffing the tree (i.e. walking at least a level of the VDOM.
0
OvermindDL1
Where id is not the element ID to update, but is rather the unique name of the dynamic data to update, then the client would have a mapping of the dynamic data names to many DOM ID’s (many being however many times it was used in the template) to then update, baked into the page.
Nope, no VDOM at all, no walking. The javascript will have to be ‘smarter’ to know the semantics of how to insert some code, but offloading that work to the client is more scaleable for the server and can significantly reduce network load, especially when combined with piecemeal JSON updates of the passed over data itself.
1