Any suggestions on a HTML/CSS/JS TreeView that fits into the LiveView environment

Phoenix Forum Questions / Help
, , ,
1

Hi,

I have some hierarchical data, think topics with recursive sub-topics, or folders and files, or an inbox with folders or, well, I think you get what I’m saying, which I would ideally like to show as a tree. (Actually, almost all my data is indefinitely hierarchical and I’m managing that with streams. It’s only a small (overview) portion of it I wish to show in a tree).

I’ve not had much luck yet identifying a suitable TreeView component to use. And by suitable I mean there are choices aplenty with a variety of strings attached in terms of additional libraries etc. Phoenix supports TailwindCSS and I have a TailwindUI license, but there’s no tree in sight there.

Any advice for me, or related experiences (positive or negative) experiences you’ve had going down the rabbit-hole of mixing React, Vue and/or another framework with TreeView options into a LiveView project? My JS isn’t terrible but I much prefer the Phoenix and LiveView way which is to essentially write the UI and the Business Logic in the server language environment. I’m afraid that once I cross that line where I need to manage another framework’s dependencies I’m entering a world of pain. That’s because my production environment is Kubernetes so I build using a Dockerfile. It works well but I’m worried about introducing dependencies and/or a different build chain I don’t yet know how to manage.

Thanks in advance.

2

If you can’t do it with CSS or a small hook, try a vanilla js component. I would avoid adding a frontend framework to get a component for as long as possible (ideally forever).

2 Likes
3

That (indefinite postponement of some JS framework) is my ideal too. I’ve seen a decade old pure css tree view around and I recall once seeing something similar which used outline, border or box shadows to draw the tree lines, but last I saw none of them really looked all that solid. For someone with a bit of time on their hands and the right (CSS) skills it does not seem to be that big of a deal, but I don’t have much of either.

Any examples of what you’d call a vanilla js component to point me to?

4

Some advice from having actually done this:

If your UI has any decent amount of complexity (beyond absolute basics), you should ditch streams and just render declaratively (like normal). The imperative API of streams makes complex UIs intractable very quickly, and the problem is dramatically worsened when the elements are arbitrarily nested. I plan to write a blog post about this soon.

If your tree is large (100s to 1000s of elements), you will run into performance issues very quickly because LiveView is not currently capable of diffing arbitrary trees (it just sends the whole thing back down the wire). You can hack your way out of this by abusing LiveComponents as they diff their own assigns. So if you build a tree of LiveComponents and pass all of the values from tree node into each LiveComponent as assigns, you can coerce LiveView into diffing the tree. This might sound inefficient, but it works well.

While benchmarking a tree with 1000+ nested nodes, LiveView updates were sending nearly 2MB down the wire for every change anywhere within the component. When my component was loaded it dispatched a bunch of jobs to update some cached values, which resulted in every node being updated one after the other (1000 updates). Some simple math reveals that this resulted in a 2GB update to the page immediately after load. The page crashed :slight_smile:

The large diffs were also very expensive to apply, and the diff latency (just in JS, not network) was in excess of a few hundred ms. You could feel it.

Implementing the LiveComponent strategy mentioned above brought the updates down to a constant (a few hundred bytes IIRC), regardless of the number of nodes. So, it works!

If you’re wondering about the use case: as I’ve posted about before on here, I have an RSS reader I’m working on with support for arbitrarily nested folders. I wanted to support at least 1000 feeds for now, so this was a real-world use case.

3 Likes
5

I things that is sound advice and validation of performance related issues I have anticipated almost instincively. As a result I’ve been very frugal with my streams usage, but perhaps even more significantly is that I have designed the content, though theoretically indefinitely recursive, to have an inherit clustering structure which I use to break what might have ended up as large numbers of elements into lightweight chunks I don’t mind sending down again if something inside them changes. This is all destined for a new version I’m working to take live so I suppose the proof will be in the eating sill. With some luck any miscalculations on my part with regards to this type of performance issue will come out in the wash before I deploy to the live servers. But your wardning is well-received and raised my guard against potential hazards. I’ve heard Chris mention a bug with recursive streams which he discovered worked almost by accident, but I am using them to good effect and have yet to run into the bug, though I’m keeping my eyes peeled for it. That’s all far more relevant to the general content than to the tiny subset of it that will end up in the tree view. That’s really only about the boundaries between the different content areas and by design should comprise a fraction of the core entities, so if the stream window management principles work well for the content it should work even better for the tree based overview. I also considered using more traditional bread-crumbs only but as far as managing the source data is concerned it much of a muchness either way.

6

To be clear, my problem with recursive streams is not that they are buggy (though I also remember hearing about bugs in the past), but that they make it too hard to build complex interfaces. Again, I really need to write a blog post about this topic because I suspect a full explanation would span several thousand worlds, but briefly:

Because streams are imperative, you have to know exactly where to insert a “change”. Especially when data is nested, this becomes arbitrarily difficult to figure out.

For example, my nested feeds have unread counts next to them. Unread counts sum up the tree (folders combine the unread counts of their child feeds). If one folder’s unread changes, the tree needs to be updated. Imagine trying to do this when you’re responsible for identifying each node that has changed.

Now imagine each node can be arbitrarily re-ordered by drag/drop. With undo/redo support. And each node has a bunch of attributes (like names). And also the app is multiplayer, so someone else can update these things in real-time.

Again, the above is not a hypothetical, I am describing the problems I had to solve for my app. There is no human being on this earth intelligent enough to build this with an imperative UI (streams). It is literally impossible.

1 Like
7

BTW, your app sounds a lot like the “Pipes” system from the mid 2000’s. I wonder what became of that.

If by emperical you mean something to the tune of that it is different for each session / user then I’d have to classify all my data and entire app as imperical to the core since every one of my users have their own way of looking at the data. If that aligns with your meaning we’ll soon enough know if I the impossibility got the better of me too or whether it gos the way of so many others - impossible only until it’s done.

I have spotted a number of differences in our approaches. Some of less significance than others but all dependent on me understanding you correctly about the meaning of imperical.

  1. As a matter of principle, for the sake of usibility, I do not roll change-counts up the tree. The only time a combined changes count should appear on a branch is when that branch is collapsed. This is in fact the main reason I chose to offer a tree overview as opposed to bread-crumbs indicating the path the the currently displayed content. From a user-perspective it requires too much mental arithmetic to tell when the change counts up the bread-crumbs no longer add up which would indicate the degree of activity happening in other branches of the tree. With a tree showing the activity where it actually happens it is easy for users to know where the action is and navigate there directly if they want.

  2. Even with infinitely nested / recursive content, there are very definite limits to how much of it can reasonably be presented in readable form on the screen. Based on that you can pre-emptively load a level or two deeper than what is being displayed but no more so it’s already loaded when the user navigates there, and similarly up the path as well. The buffer levels on either side and the currently visible levels should be all you ever work with and though screen resolutions have gone crazy the way it’s being used in real life still puts reasonable bounds on the amount of data you’re dealing withat any given point.

  3. Perhaps as a result of the truly challenging structure (virtually chaotic) of my app’s real-world data it was never a viable option to cycle through volumes of data for any reason, finding changes or otherwise. None of my data is external and even seemingly free-form content is structured. I suspect that when your frame of reference is the likes of news feeds that seems incomprehensible but it is a cornerstone of my app that’s been decades in the makeing. The impact is that I have full control over how and when I assign and use their identities and even then there’s almost never a long list of elements at the same level on the same branch. That makes stream operations unambigious, which I sense is where you’ve enountered some of your issues.

  4. I do intend to support some drag and drop in my app (I don’t yet) but have’t even came close to considering it a means for users to reorganise the contents of the trees they see. I simply don’t envisiage that type of change ever becoming "legal " in my app. In essense, the bulk of my data (specifically as soon as any other user has interacted with it) becomes something akind to immutable, meaning they are no longer subject to change except by adding a new version linked to the original. The prospect of not only changing the content but the entire structure of the data by someone dragging and dropping is well outside the realm of what I consider feasible. If it is desired and demanded, the functional equivalent thereof can be implemented by adding references to the data in other parts of the tree, silimar to symbolic links in file systems, but the “move” operation is not supported. Well, technically it is, but only as long as only the original author has control of the data and is still prepping it for a wider audience. The moment another user gets to see it it can only be changed by reference to a replacement.

I don’t know how much of this makes sense to you, and like I said it remains to be seen how well it works out in practice. It’s good to know though that there are others out there grappling with similar real world issues in production systems with experience and helpful advise if things if or when this impossible task I’ve taken on overwhelms my intelligence.

1 Like
8

By “imperative” I mean that the Streams APIs force you to manually instruct the DOM which parts have to be updated. As opposed to “declaratively” handing the entire, newly-rendered DOM to the browser and using an algorithm to automatically diff it out.

If you’ve been doing web dev long enough to remember the “before-React” times, what I’m describing is essentially the difference between React/Vue (declarative) and the JQuery-style approaches that preceded it (imperative). In the old times, to update the value in a div you might, say, call document.getElementById(...).innerText = 'whatever', whereas in React you instead render the whole thing up front and the diffing engine figures out what to do.

The declarative (React) approach is much, much better, which is why everyone uses it now. Including LiveView - this is how LiveView works by default! Until you use streams, and then you’ve essentially stepped back 20 years to the “old times”. This is why I don’t like streams.

Again, this topic is really too much for a comment (it would take a whole article to explain), but this is not the issue I’m describing. The problem is inherent to the stream APIs themselves: you have to specify exactly what changed and where. The problem is that complex applications are like double pendulums: they are chaotic. One small change somewhere can result in many, many changes down the line. I offered my unread counts as an example because they illustrate the problem well:

If a user opens a post, that post is read. Therefore, the feed’s unread count changes. Therefore, the unread counts of all parent folders also change. If you’re not using a declarative API, you have to write code to change the parents’ unread counts.

Okay, now let’s make it worse: a feed is moved to a separate folder. Now you have to recompute the old folder’s counts, the new folder’s counts, and all of their parents. You have to write more special-case code to handle this too.

Now someone else moves a feed. Or a feed is deleted. Or un-deleted. Repeat ad infinitum for every single feature you add to your app. This is intractable.

There is only one way to avoid this with an imperative API, and that’s to avoid adding features to your app. So, apps built in this way are not very good! That’s why I don’t like streams.

1 Like
9

Tbh I feel like you don’t like streams because you’re working one a usecase streams are not meant to handle. What you’re describing is imo indeed solved better by not using streams, but that’s not the fail of streams.

Streams works quite well for the usual chat or other infinite scrolling usecase where you generally prepend, append or update by ID on a single level. Once data is nested streams is no longer the API to use.

5 Likes
10

I somewhat agree, and like I said I’m probably still 10,000 words short of a fair and thorough criticism. I will save that for a blog post down the road :slight_smile:

I agree of course, but the argument I want to make (down the road!) is essentially that the hierarchical use case is simply demanding enough to reveal a fundamental, underlying problem. The problems with streams still exist within those simple use cases, but they are simple enough that it just doesn’t hurt that bad.

But therein lies the problem: once you start to architect your app this way, you are trapped in the simple use case. To escape, you must transcend the imperative APIs. There are cases where this might be okay! You could essentially treat it like tech debt and redesign down the road. But I think a lot of people will end up not knowing how to progress because they don’t understand the limits of the APIs which were recommended to them (streams).

(I would also like to point out that the OP of this thread was doing exactly that: using streams for a nested use case - I felt the need to post precisely because I have felt that pain before.)

Again, not even close to a thorough criticism! :slight_smile:

I would also like to cite my favorite programming article (ever), which essentially contains the core of my argument (but in a general case):

Clearly I am making a habit of posting his work on here, but perhaps that is not such a bad thing…

1 Like
11

I am clearly failing to resist the urge to keep writing about this, but I just want to illustrate what I mean here:

You’ve built a simple chatroom with LiveView and Phoenix PubSub. Great!

You add the ability to edit messages. When a message is edited, PubSub kicks in, and Streams updates the message in the DOM for all users. Simple, easy!

You add the ability to reply to other messages. When a user replies to an old message, the original is previewed above the reply (a standard feature).

Now, the old message is edited. You now have to special-case your Streams code to update both the original and the reply. We’re one feature in and it already hurts!

This is just one example of a totally standard chat feature. Try to think about how much more trouble this could cause you as you build out more functionality. It is a real problem.

But the solution was discovered over a decade ago: it’s called React, and LiveView implements the React model by default! Except for streams, which break it in exchange for performance. This is dangerous.

12

I would argue against changing the quoted message in this case. When I reply to an old message, I meant to reply to that message as frozen at that time, or my message can be totally sabotaged.

I agree with you in general though. Streaming is not a panacea, it will cause more harm than it is worth if you try to be too fancy.

2 Likes
13

We are bikeshedding here, but it depends on context. In a chat app, an informal setting where “old” usually means a few minutes ago, updating the message is the correct behavior. As far as I know all popular chat apps behave this way. To be clear, I was not referring to a “quote” feature but a “reply” feature - these are different things. Quoting is what I just did to your message - totally different. The purpose of displaying a “reply” is to provide a preview and link to the previous message. Try it in your favorite app, and you will see the behavior I described.

But the specific feature is not the point. Think about this:

What happens if the user who posted the first message changes their name? Their profile picture? What happens if a third user comes by and changes the color of the role the first user has applied? These are all real features!

What happens if a user reading the messages goes into their settings and turns off reply previews? Which messages do you update then?!

This just goes on and on, forever. As I said before, there is no human being on this earth smart enough to write all of these special cases without bugs. It is literally impossible.

1 Like
14

That is a judgemental and grossly misguided sentiment that should not remain unchallenged. The features you add to your app should be solely informed by what value users get from using it, not what is easier, quicker and cheaper to implement or makes for the flashiest demos. If the features your users actually need and ask for require that you completely rethink how you approach the implementation, then that’s what you do. That’s how barely adequate apps evolve into good apps.

You mention of developer intellect, but that’s not the limiting factor you should be worried about. If with the accuracy and perfect memory of computers, the vast combined computing cycles of your servers and client’s UAs your the algorithm you (in your imperical model) or your platform provider (in your declaritive model) is using battles to keep track of the changes happening, your users, who have to do all that mentally with material they are only vaguely familiar with and is too volumous and complex to fit in their heads at any given time which is (probably/hopefully) why they’re using the app in the first place, will have lost interest in even trying a long time ago. It strangely comes back to one of the oldest truisms in computing - do not try to automate something for which a manual process does not exist. I think that’s being ignored far too often these days. It simply means that it should be known long before you start writing your code how each of the allowed/possible changes should be handled. Without that you’re guessing and end up using brute force.

Even if we completely disregard malicious intent and plain stupidity, the moment you’re going to expose me to such volume and complexity of content which one or more collaborators can manipulate wholesale at will, is the moment I’ll simply close the app, write off my investment in it as a bad experience to learn from and go find a better solution. And that’s if you’re lucky to only have users intelligent enough to realise they should. The rest will haunt you with complaints and support queries until nobody can afford to contunue and everyone loses.

A halmark of intelligence is how it tends to simplify complex matters. A halmark of stupidity is how it complicates simple matters.

I have empathy with your desire to lay out your whole argument in such an article, and you may as well do it. But I doubt that would serve the communal interests. Obviously @chrismccord considered streams a critical element of LiveView 1.0 for some better reason than maintaining backward compatibility with 20-year old thinking. Rather than going on a rant against streams in a powerfully written article (Is Ash a backcronym for the Anti-Stream-Huddle?) which may leave the community divided and confused, you, Chris and anyone else involved should have it out in a pointed but frank discussion aimed at identifying the pros and cons of streams in different scenarios and making sure you and others are using the technology for what it was written for rather than to regress to ancient approaches.

My personal take on the core issue you’ve been facing is that you’ve failed to incorporate ways of reducing the volume and complexity of the data you’re presenting to the user. Somehow you thought that it would be a good selling point to say you’re managing 1000+ new feeds but that is of no use if you’re still making the volume and complexity the user’s problem to contend with rather than making it ludicrously simply for them to grasp what’s happening in those feeds. Add to that the notion that allowing easy ad-hoc changes to shared views on how these feeds are organised might be a nice feature to add to the marketing material and you got a recipe for a truly technocratic product - i.e. a solution looking for a problem. That’s my 2c, use or don’t, it’s no skin of my back. The fact or possibility that you’ve ended up with intractible complexity because you taken a poorly conceived product to market does make what you described a real-world problem you’ve had to deal with, and I feel for you. Hopefully we can learn from your mistakes, which might or might not have included using or misusing streams, but which it is will come to light if or when you and Chris have your big debate.

15

I appreciate you intentions and afforded you inputs a great deal of consideration. My conclusion is that streams are less to blame for your pains than your expectations and app design.

By the very nature of computing both what you’ve described as imperative vs declaritive code results in sequences of instructions being repeated and branched until the job is done without exception. You either write the code yourself, declare it to be an instance of a use-case someone else has written code for or most typically a combination thereof. But somewhere along the line somebody with a grasp of the problem at hand has to implement some form of algorithm to solve it. Most of our jobs today is about modelling the data and required operations of some problem domain with the intent of allowing tried and tested algorithms to perform their magic on your domain’s data. Until you’ve adequately and accurately modelled your problem domain nobody’s algorithms will deal with it correctly, not even your own.

Declaritive programming is brilliant. Despite there being even more declaritive alternatives (I hear Ash making such claims) in many respects Erlang, Elixir, Phoenix, Ecto, LiveView and even streams are all in fact already more declaritive than (what I would call) procedural (or you’d call imperative). But there’s always a catch. With declaritive setups it’s that they lean heavily towards being (I believe the term is) opinionated. That is, they work really well but only as long as their imputs conforms to the use-case(s) they were designed for. If what you present to them is even a little off, you get no result (if you’re lucky) or a result you didn’t expect such what you’ve described. But that’s just it - it’s unrealistic to expect any framework to solve problems for you when your own grasp of your problem domain isn’t sufficient to judge which tools are or aren’t geared for the solution you need and more importantly how to structure the model you present to it to enable their version of the solution. In essense, to implement a solution you need to first understand what the solution is. You can then try map your version of the problem onto how pre-written solutions need it or you can build your own using component solutions you are familiar with. If you don’t care where you’re going you’re bound to get there. If you don’t know what algorithm(s) are required to solve your problem you’re bound to find problems with the ones you try to implement.

Either way, I remain vigilant in my efforts to make smart choices about how I use what tools, but after careful consideration of your advise against using (nested) streams I believe my approach is on the right track after all. I’ve designed my application from the ground up to accurately and succinctly track, handle and present users with real changes in their most simplified form. I that regard I’ve found a high degree of correlation between the principles both LiveView and its Streams are based on and how I’ve modelled my applicaiton and its data.

16

So do I, but with a different comment. I also agree that as far as replies and quotes in messaging apps are concerned defence against the dark arts is often a bigger concern than correctness. However, let’s for argument say we’ve decided that we want a preview of the edited version of the replied-to message to be shown above the reply? If you have your data modelled so the replied-to message is referenced in the reply structure the job is already done provided the component you use to display the message is given the in-reply-to message and knows to display it if present. If by contrast you’ve gone and made a copy of the original and stuck that in your reply structure you’ll obviously experience pain dealing with it. Next feature? I bet that too will either be a non-issue because it was anticipated in the overall design or the design is agile enough to accomodate the required changes to implement the feature, or it will be a massive PITA to implement the feature as special cases in the wrong part of the application.

Feel free to blame it on your Systems Architect, a role I’ve excelled at for most of my career. As such I was accountable for ensuring the data- and business model make it possible if not easy to add the value the users expect which include all the features they ask for. It’s common for people to handle your own architecture, but you know how it’s said that a lawyer representing themself has a fool for a client, right? That said, I am my own Systems Architect right now, so I too am that fool. But I’m also my own boss and my own pool of programmers, designers and graphic artists, so if I get my models wrong “we all” suffer the consequences.

17

Where does ash come in to this? :sweat_smile:

18

I know nothing about Ash beyond a few remarks made about it on this forum. One of the remarks was about it not being a replacement for Ecto but a declaritive upgrade in honour of it. The remark was in response to someone going on about declarative vs imperative coding and how bad streams are so welcome to my mind getting provoced into the weirdest connections like that.

The little I read (as in no surface got scratched in the process) about Ash left me with two points in summation.

  1. It looks like it might be extremely well aligned with my natural model-centric mindset, so much so that I’m virtually certain to get sucket right into its vortex if I got too close.

  2. I’ve invested too many resources in aligning my design and approach with Phoenix, Ecto and LiveView to change tack right now. I’m one single person doing everything from vision to hardware and I cannot afford to stretch myself out any thinner.

I’m well aware how much I am behaving like the General preparing his bowmen and foot soldiers for battle being approached by a man saying “Hang on, I invented this thing called a canon that shoots iron balls at high speed at the enemy that will win you this battle within the hour!” The General dismissed him angrily with the words: “Not now, you fool, can’t you see I’m fighting a war here?”

I am looking forward to having the opportunity to discover what Ash is all about and how best to seize its opportunities and tell myself that at least it will be that much more settled and mature by the time I have that luxury. It was a lot like that for me with Elixir and Phoenix. When I started my current project which is also my life’s work, Elixir didn’t even exist. I knew the only viable way to write the server I need would be in Erlang, but because it was such an intensive endeavour I postponed writing “the real” server and implemented a makeshift starter-version in PHP with Laravel. Got lots done but when I eventually couldn’t make useful progress I switched tack to Phoenix and Elixir and replicated most of what took me a year to build in Laravel within a couple of weeks, and I was making such great progress in the new environment that I completely readjusted my goals to include a proper server right off the bat. It was a good experience, a great call but the perfect thing about it was the timing. With any luck I’d get the timing for Ash just right as well.

19

Understood. Well Ash doesn’t aim to be a distraction for folks who have their tools on lock without it :grin: any tool should be adopted for pragmatic reasons (I.e “I need X property or capability that I don’t have”). If you don’t feel that way, no need to make a change :sweat_smile:

Personally, any principle exists to break, and I often see folks pin declarative and imperative as polar opposites of each other whereby something is “either but not both”. It’s a spectrum. You should make something as declarative as is reasonable to make it. The new ash installer is mostly declarative data that is held together by grug-brain-level-of-abstraction-simple vanilla js.

Ultimately, your declarative code is someone else’s imperative code. The computer gets instructions some way or another :joy:

An argument could be made that liveview streams are imperative and/or declarative, it all depends on your perspective :grin: you aren’t hand crafting websocket messages after all.

3 Likes
20

I’m, uh, not exactly sure what happened to this thread while I was away, but I will try to address a couple of things.

First of all, it’s clear to me that my choice to abuse the terms “declarative” and “imperative” with respect to frontend may have created more problems than it solved. These are very overloaded terms in software, so I apologize for any confusion.

By “declarative”, in frontend I am referring to the “React model”. You can go back and replace “declarative” with “react-like” and “imperative” with “jquery-like” in my comments if that helps. The terms “immediate mode” and “retained mode” are also sometimes used in this context, but they don’t mean exactly the same thing as what I intended so I avoided them.

I have absolutely no idea how you got dragged into this, but just to be clear: I have never used Ash framework before, and I don’t know much about it. However, as I understand it’s mostly a backend framework, and its use of the term “declarative” is completely orthogonal to my comments about frontend and LiveView.

1 Like