Drut: an idea for the low-level foundation for LiveView, Texas or Drab - like libraries

Hi all,
based on this, and discussions here before (like @OvermindDL1’s idea to use cookie to transfer the token), there is an idea to create a low level library which may be a foundation for Drab and other guys.

It should not be based on Phoenix, but rather on Plug and/or Raxx (extendible for the future), so you can use it in other frameworks as well. It should have the own transports, websocket for the beginning, but extendible as well.

The API I am thinking of is rather straighforward, as in fact Drab is build over few functions:

  • exec_js - runs given javascript in the browsers; should be both sync and async
  • broadcast_js - runs given javascript in all the browsers listening on some topic; only async
  • exec_elixir - runs Elixir function from the frontend; sync and async (TBD)

exec_elixir may be replaced with something more message-like, rather than running the function with its name (just like @chrismccord did in the LiveView demo), TBD. Security: TBD.

Everything on the upper level may operate on that. We can move Drab to use Drut in Drab 2.0 (it will require changes in installation process, etc). Some parts of Drab (Drab.Element, Drab.Browser, even Drab.Query) might be pulled out of Phoenix, but some must stay (Drab.Live strongly depends on Phoenix engine). Things like Commanders would stay on the Drab level.

WDYT?

8 Likes

This sounds like a great effort! As Drab is approaching 1.0 it makes sense to start looking at the abstractions and building blocks that could be extracted. However, I caution working to strip things down too early. For the LiveView case, it’s definitely too early for us to start drawing the abstraction lines, since a lot of those abstractions we are still working to fully figure out :slight_smile: Extraction is good where it makes sense, but going too early or too granular can slow you down or be a big maintenance burden, so I wouldn’t be too quick start stripping Drab down prior to 1.0. As things progress on the LiveView side, I’ll be happy to revisit common abstractions, but it’s too early on my side at the moment :heart:

8 Likes

Well, this would be more the side project, rather than stripping down Drab, but they should meet at the some point in the future. What I want to do is a kind of Plug for a server-side frontend processing, which anyone can use to do something like Drab, on any Plug or Raxx based framework.

I realize this is a big effort, as there are many things which Phoenix does for free, like PubSub. But wait, we can use PubSub as it is already extracted from Pheonix :slight_smile:

The future of Drab itself is still unchanged, I will release 1.0 soon.

Things like Drab.Live, LiveView or Texas are Phoenix-based and obviously need to stay in the Phoenix world.

5 Likes

Hi Grych,

Really appreciate all the work you have put into Drab. You gave us a fresh perspective on how the client-server relationship could work.

Since you are now at a crossroad and need to decide how to proceed with Drab, may I suggest something radical… Please look into the possibility of building a complete web framework using your ideas from Drab and more.

What would an alternative Elixir framework look like ?

  • How about a framework with no DSL? The syntax should look and work like plain Elixir code. Most of us came here because we love Elixir. But some of us prefer not to learn additional syntax, especially if it does not look like Elixir. Your framework can still leverage all the Elixir goodies like pattern matching and piping - but no extra DSL and no magic under the hood.

  • As you mentioned you could use CrowdHailer’s Ace + Raxx for the underlying webserver and HTTP stuff.

  • What is the most valuable thing in the Elixir/Erlang ecosystem? OTP ofcourse. But OTP has un-necessary wordy syntax and just seems out of place in Elixir. Perhaps its the one part of Elixir that still looks like it came from planet Erlang. Using something like ExActor as the default could smooth those edges for newbies.

  • We need some kind of ORM or at least a way to connect to Postgres. Maybe look into Moebius for ideas - it has a functional chainable elixir like syntax. https://github.com/robconery/moebius

  • We need something like channels. Perhaps something which works like sending/recieving messages to/from an Elixir process at a lower level. At a higher level the channels could be compatible with GraphQL + Subscriptions. (Absinthe)

  • Perhaps Drab could be modified to work as a new abstraction, which extends the GraphQL syntax. I told you this is going to be radical…:smiley:

  • Throw in an optional, opinionated, Elm-like front-end stack which works on Vue.js + Vuex + Vue ORM. If someone doesn’t like Vue, they can plug in something else later. By having GraphQL as your default channel mechanism, you open up yourself to all possibilities.

  • Complete the stack with a cross-platform mobile development sdk - Nativescript-Vue is just reaching maturity with version 2.0.

  • And oh yes, if you like these ideas, we can look into better names for the framework and sub-components. Need to do better than Drab (means plain and boring) and Drut (??)

For those of you who are happy with Phoenix, please ignore this post. Phoenix is an excellent high performance web framework, and Chris is doing great work with advancing the frontier. Some of us have minds which view the world differently, and so we continue to seek alternatives.

This was just a brainstorming exercise imagining an alternative Elixir web framework.

4 Likes

So more or less Ember, but built on top of Ace + Raxxx + Drab + Vue.js … ?

3 Likes

I love the idea of trying to suss out the common building blocks for libraries like these, but I think texas is so fundamentally different than drab (and likely liveview from the discussions I’ve seen) it’s just not likely to have much in common other than the high level problems were trying to solve. Texas doesn’t intend on ever giving the user access to pushing JS to execute on the client and as a core goal (at least for now) texas wants to do as much execution on the server as possible to give the users of the library the ability to always control their runtime (as much as possible, there will always be some client-side execution)

Texas caches the dynamic fragments and keeps that cache on the server so it’s only pushing up minimal patches. I believe drab is pushing up data in the assigns part of the conn, right (well specifically it’s not using the conn struct, but it’s using variables that are accessed the same as if they were in the conn)? Now we both might be able to diff and only send up patches (not sure if drab is doing that currently?) but texas will be diffing against AST rather than variable data (such as you would find being passed in with the assigns) so even the diffing algorithm would be substantially different.

4 Likes

if you find some core patterns there I’d love to hear about them - I don’t want to discourage you from looking into it, but from the research I’ve done - they are very different solutions to solve the same problems

2 Likes

with all of that said - I hope some of the people using drab take it upon themselves to make drab more pluggable and isolated from phoenix - you’ve built a solid foundation around some amazing tooling and I wish I’d known about drab sooner and I likely would’ve been contributing rather than writing my own library!

I think there’s huge merit to isolating our libraries from any specific web framework, but as you know phoenix just provides so many great building blocks it’s much easier to get our 1.0 implementations working on top of phoenix

2 Likes

Yeah, I also don’t believe you can find many common abstractions between Drab and what I think LiveView will be. But you can try to implement a LiveView clone on top of Drab if you can convert diffs of Elixir values into pokes. It sounds like it would be super efficient.

3 Likes

I have heard of Ember, but I am not very familiar with what it does differently. Checking it out now…

I suggested Vue.js because it is one of the JS frameworks that offers close to native performance when developing mobile apps; the others being React and Angular. Vue and React are both popular in the Elixir community. IMO, Vue offers a more crisp, less Javascripty syntax compared to React.

3 Likes

Shortly: nope.
I don’t see the space for Yet Another Framework. And I am not sure I can do it, in terms of time an resources. We don’t want to end like Volt, don’t we?

I am very attached to those names! :slight_smile: They are short, easy to pronounce and meaningful. There are other languages, you now :slight_smile:

4 Likes

True, you don’t need Drut for Texas, as Channels are enough to push the changes back to the client. And because how it is done (operating over the template), Texas (and LiveView, and Drab.Live) with stay in the Phoenix world, no doubt.

This proposition is way more low-level, and thats why I wrote before started writhing. Maybe it is too early for such project? Maybe it is too much to re-write the great Phoenix Channels? Maybe it is too much to give developers exec_js instead of traditional websocket messaging? (did I just call websocket messaging traditional?)

Yes, Drab.Live only sends the small patches, but it is not doing the diff, but identifies where to change the page by marks which are done during the compilation process. The idea of diffing on the server is a way better I believe, and do not need a special engine. I just didn’t think about it, as Drab.Live evolved in time (for example, at the first step, it did not re-render the whole template, but rather executed small chunks of AST stored during the compilation).

Anyway, under the hood, Drab.Live sends the patches using exec_js. This is the part I want to isolate.

Absolutely true. But now, when I am reaching to graal of 1.0, it is a time to think about the future, this is why this post exist :slight_smile:

I imagine one could write a framework (something like what @raza described above), based on Drut, but with the different approach to communicate back to the browser - from “sending messages to the browser” to “executing code in the browser”.

5 Likes

I’ll definitely be following any progress you make. I’m looking forward to seeing what all you come up with!

4 Likes

Yes, but I can find the abstraction between future Drab and the future LiveViews :slight_smile: Now, if you want to do Drab or Texas on the top of the other frameworks, you must implement socketing yourself or use the specific for the platform.

I do remember discussions here long time ago, when I introduced the first proof-of-concept of Drab, based on jQuery. First question was: can you get the jquery part out and leave the exec_js only?

3 Likes

/me coughs ^.^;

2 Likes

I think this is a great idea although sadly I am not informed enough to claim how viable or desirable the separation would be.

IMO a huge first step would be to somewhat emulate what React does and only send diffs. Where can I read in more details about what Drab is doing right now? I don’t mean the intro / tutorial but specifically the part on how does it decide what patches to send?

The idea of Drut is way lower level, to provide a common communication wire, regardless of the framework.

About sending diffs, Drab is already doing something similar. This is described in the documentation.

2 Likes

After trying to implement my UndeadView, I now agree completely. I’d end up duplicating a lot of what Drab already does. There might be a role for Drut after all. Or maybe Drab should concentrate on becoming Drut?

And maybe the most important thing is to abstract over the differences between HTTP and channels where possible.

2 Likes