Client-side rendering, server-side state (StateChannel)

hubertlepicki

I have been thinking a lot about what are the pain points in web development, and what the remedy can be. First off, I’ve been programming web applications since about Rails 1.2, when the REST concept was introduced and was considered revolutionary, and I went through different stacks, that were both server-side, and client-side rendered: Dojo, Ember.js, React, Stimulus, LiveView, and many others…

Recently I was taking part in several projects using LiveView and one using Surface UI. I have got to say that both are great and definitely step in the right direction because they solve the problem of the biggest pain point in web development: creating and consuming the APIs. They solve it by cutting out the middleman completely, allowing you to fetch the data in Elixir, then, in a place where you’d have to build API and API client - do absolutely nothing! - and then render the template. Boom, instant productivity boost.

But LiveView is also not without it’s trade-offs, and a major one is the fact of being… Elixir library, and not JavaScript library, so the integration with JavaScript ecosystem (via Hooks or otherwise) is not always as easy and straight-forward as it’d be if I just rendered the templates in React. There are other issues, obviously, that I won’t mention here, but everyone who worked with LiveView on a fancy, JavaScript-heavy app knows this one so it’s what I mention.

Now, what if we reversed the situation? What if we kept React (or your framework of choice) and render the application client-side, in JavaScript, but we also kept the beloved feature of LiveView and not have to write or consume any API?

My idea is to put the state in the PhoenixChannel, and sync it with the client on the JavaScript side, whenever it changes.

The idea is that the client-side code emits events, that are being sent to Phoenix Channel as messages. Phoenix Channels library already is taking care of making sure the events are arriving and being processed in order in which they were emitted. Once they reach Phoenix Channel, they get consumed by programmer’s code and state is being updated, and synced back to client-side JavaScript app. The client-side app (say in React) reacts to state being updated and re-renders the parts of the application that changed.

I was experimenting in how to do that, and came across JSON diff and patch, as described in RFC 6902 - more info here: https://jsonpatch.com/.

There are libraries for both JavaScript and Elixir that can compute diff of arbitrary JSON structures, and apply patches. Patches are quite small and concise, and understand appending, inserting at position, replacing, and deleting parts of JSON/nested maps. It’s also relatively cheap to compute a diff, and apply a patch.

So, my thinking is that, the Elixir programmer would initialize state in the Phoenix Channel, to a map, map of maps and / or other basic types, and either update the map directly - resulting in the library that I created to wrap Phoenix Channels to calculate the diff - or - in case of large state and the need for speed - the Elixir programmer can produce JSON Patch operations themselves, and send these over the wire directly to the client-side app and apply patches locally on the server-side, so both states stay in sync.

The relevant Elixir code that does the above, and wraps around Phoenix Channels can be found in this file state_channel/lib/state_channel.ex at main · amberbit/state_channel · GitHub

On the JavaScript side, I chose React, but it’s possible to create bindings for pretty much anything. Anyway, on the React side, we have a library that delivers Providers for Channel and State, allowing for the React components to just consume the state, without worrying about it’s synchronization, and allowing for events to be sent to the backend. The relevant file with JavaScript code is here: state_channel_react/src/index.jsx at main · amberbit/state_channel_react · GitHub

Now, how would a programmer use it? Let’s start with a React application. At first, the programmer needs to wrap his or her components into a provider, that connects to the specified channel and listens to the state changes, like this:

    <PhoenixSocketProvider>
      <StateChannelProvider topic="app:state">
        <MyComponent />
      </StateChannelProvider>
    </PhoenixSocketProvider>

Within the component, you can consume the state, pass it whole or as props to child components, and emit events:

const MyComponent  = () => {
  const {state, pushMessage} = useStateChannel();

  return(
    <div>
      <h4>Current state is:</h4>
      <pre>
        { JSON.stringify(state, null, 2) }
      </pre>

      <a href="#" onClick={(e) => {e.preventDefault(); pushMessage("change_state")}>Change state!</a>
    </div>
  )
};

The attached click handler sends message “change_state” to the server.

Now, in our channel, responsible for handling “app:state” topic, we can react to the state being changed, by setting a new state. The state diff will be then computed, and JSON diff structs will be transfered over the wire back to the client, and it will re-render using the new values.

defmodule MyApp.AppChannel do
  use StateChannel

  @impl Phoenix.Channel
  def join("app:state", _message, socket) do
    {:ok, socket |> assign(:state, init_state())}
  end

  @impl Phoenix.Channel
  def join(__otherwise, _params, _socket) do
    {:error, %{reason: "unauthorized"}}
  end

  @impl StateChannel
  defp init_state() do
    %{
      "greeting" => "Hello, World!"
    }
  end

  @impl StateChannel
  def on_message("change_state", value, socket) do
    socket
    |> set_state(%{"greeting" => "Hello, Elixir!"})
  end
end

Again, this works with small state well, for large state you can use patch_state with JSON path that will update the server side state in place, and send the diff - that doesn’t have to be computed - to the client:

  @impl StateChannel
  def on_message("change_state", value, socket) do
    socket
    |> patch_state("/greeting", "Hello, Elixir!"})
  end

Since state updates happen exclusively on the server, in Channels, this approach can be used to build collaborative applications similar to LiveView, or listen to some server-side events and update state, or even components on the same page can communicate using OTP messages or PubSub to interact behind the scenes - and when state of either of the components gets updated on the server - it gets synced to clients.

I have put together a little bigger demo and deployed to fly.io:

https://react-phoenix-server-state.fly.dev/

The source for the app is here:
GitHub - hubertlepicki/shopper_umbrella: An example application to demonstrate StateChannel + React · GitHub and releavnt files are:

front-end React code: shopper_umbrella/apps/shopper_web/assets/js/main.jsx at main · hubertlepicki/shopper_umbrella · GitHub
back-end Channel code:
shopper_umbrella/apps/shopper_web/lib/shopper_web/channels/app_channel.ex at main · hubertlepicki/shopper_umbrella · GitHub

The sources for React and Pheonix libraries used by the app above are here:
https://github.com/amberbit/state_channel_react
https://github.com/amberbit/state_channel

The code itself is a proof-of-concept quality, but, for the task of the demo app - it seems to work just fine.

What do you folks think of this approach? I am sure there is a thousand concerns, and things I didn’t consider, and I would appreciate suggestions and criticism.

37 comments

/phoenix #channels #react

11 5138 37

2023-01-08 10:13:01 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

kanishka

You may already be familiar with this, but here is a related approach with similar goals - How It Works - Inertia.js Documentation

Post #1

Sebb

I’m a little busy right now, but as this is very exiting a few quick thoughts.

I’m thinking about the exact same thing some time now. Then I found Storex, see Storex - Frontend store with the state on the backend - #8 by Sebb and played a little with that. Do you know it?

I have some demo code where I rebuilt the solidjs-store example
https://www.solidjs.com/tutorial/stores_createstore?solved

in Elixir.

The only problem right now is, that the Storex store does not play nicely with solid’s reconcile and from - one has to clone all data coming in. But its good enough to see that this is a great approach.

Very much looking forward to what becomes of this, and eager to help, will look at your demo asap, ie next year

Post #2

Sebb

interesting, but a GenServer-based backend (so basically a LV without heex but reactive rending from a Elixir controlled client-store) will surely have the upper hand here in theory.

Post #3

hubertlepicki

Wow, I did not know of Storex, and @kanishka , I didn’t know about interia.js. Thank you both!

Storex (//cc @drozdzynski ) looks super similar, especially in usage, but I would be super interested to hear why @drozdzynski you chose not to use JSON diff/patch and implemented your own solution.

Post #4

hubertlepicki

OK so I did some comparisons with Storex and deployed it here, you can compare the same app / UI:

StateChannel + React:
https://react-phoenix-server-state.fly.dev/

Storex + React:
https://react-phoenix-server-state.fly.dev/storex_test

The code I translated to Storex is here:

https://github.com/hubertlepicki/shopper_umbrella/blob/main/apps/shopper_web/assets/js/main_storex.jsx

and

https://github.com/hubertlepicki/shopper_umbrella/blob/main/apps/shopper_web/lib/shopper_web/app_store.ex

When it comes to usage, performance and such, they’re both very comparable. StateChannel uses less text over the wire, as you can see below, but not sure how significant is that.

In general, I wish I knew about Storex before I started this thing, but also it’s great to see the code converged to very similar solution for the problem, independently, twice.

Post #5

hubertlepicki

I’m thinking this approach would work really well with server-side rendered JS apps too, if I add support for exposing the initial state of channels through API. Then, this API could be called from the server-side part of the app to fetch the initial state and render the page server-side. With Next.js, this would be done using getServerSideProps: Data Fetching: getServerSideProps | Next.js

Then, we could do either what LiveView does, and initialize the state again when the Websocket connects or do a slight optimization and hold the state in memory for some time until client via websocket connects and then hand over the state. I suspect the first approach (like LiveView double render) is both easier and safer.

Post #6

Sebb

Now I have some time to look into this, some quick thoughts:

As I said before, I completely agree with your premise. We need sth like this.

There is even more prior art, see

So obviously there is something going on. Forces should be joined.

Good point. Nice and straight forward impl with set/patch_state. This should make the backend fast enough (and faster than storex).

As for the client-side impl, can you say how busy React is applying the new state? For solidjs there should be a way to directly translate a JSONpatch to setStore’s path API (so there is close to no work to do on the client).

SSR is interesting, but I have no clue and no comment.

What do you think about using Phoenix routing with StateChannel? Or generally speaking: what is needed to do with StateChannel and Phoenix what https://inertiajs.com/ does.

What do you think of implementing the GitHub - krausest/js-framework-benchmark: A comparison of the performance of a few popular javascript frameworks · GitHub app instead of your shopper. Will give some more insight in the performance of the approach.

Phoenix LiveView vs SPA

Post #7

methyl

I’m the author of tha live_data where I explored this idea, thanks for the mention @Sebb.

I still think it’s an interesting direction. The most problematic thing in this approach (true for both LiveData and LiveView) is that it relies on network to apply local state/UI changes. For some cases it’s not a big deal, for example if we are building a big interactive dashboard when most of the changes are going on on the server. For building apps in general, I think the ability to apply mutations locally is a must, that’s why I was never a big fan of LiveView.

Since then, I’ve discovered Replicache and watched its development closely. It builds on the idea of holding the state on the server as a single source of truth and pushing diffs to clients, but it also allows clients to apply mutations optimistically with casual consistency which results in a much better user experience.

While Replicache is (as of now) designed mostly around JS ecosystem, it’s in its core backend agnostic. I think exploring how well it will play together with the Elixir Way is worth looking into, I have a feeling it’s a great fit.

Post #8

Sebb

Warning: I have very limited web-dev experience, just some internal tools in LV and Django, so most likely I’m missing stuff. (I’m looking into this, because I see the pain we have with an Angular App with REST-Backend and how easy that would be in LV)

As I see it we have 3 kinds of state:

local state, never interesting for the server, never stored in the backend. This can be easily handled by a separate store on the client. (example: the position of an element while moving it)
computed state, always needs the server. (that’s the core use case of StateChannel et al.)
server checked and/or stored state. The state can be computed by the client, but may be invalid, optimistic render would be nice. This could be done with the same approach Hubert uses on the server: optimistically update the local state and send the patch to the server. Client will directly render but maybe the server will overwrite the change. (example: the position of an element after movement has finished. We may want to store this in the backend and it may be invalid, so the old position has to be restored).

Things get complicated if you need CRDT, which is only true for collaborative apps. Replicache seems nice for that as long as you are OK with paying for a blackbox.

Post #9

hubertlepicki

I think in real life, the lines often get blurred between these types of state, but my view is that majority of state actually should live on the client, and the client (as in web browser) should also do it’s share of rendering HTML and processing the data/state. Not only this saves us some CPU cycles on the server, that we have to pay for, but the user gets the ultimate “edge computing experience” where the computations needed are done on their own device.

Having said that, I think in the ideal web app, the lifecycle starts with client requesting a page, and, it makes sense to deliver that already rendered into HTML. Then, ideally, the client takes over and makes the page interactive, maybe whole page maybe just bits of it here and there. Then, when user navigates to another page, or significantly changes the data that is displayed on the page (as in going to "Next’ page of paginated results), another request to the backend is made and the page the user sees should morph into another page, the new elements should be made interactive, and ideally - some of the elements on the page that didn’t change shouldn’t be updated at all. I did some digging over Christmas and the framework I found for achieving the above exists, and it’s also within React ecosystem - Next.js 13 with React 18 and Server Components can achieve (and indeed does according to my tests) just that.

As mentioned in the original post, the main pain point in developing web apps is creating APIs, and consuming APIs, and this can be avoided with Next.js by fetching most or all of the data in one go, when the page loads, similar to the way we do it in Phoenix controllers.

In fact, I played some more with Next.js and was able to hack together a simple solution where Next.js is treated as a “view layer” for Phoenix: https://twitter.com/hubertlepicki/status/1607705791460528128

Then, the majority of user interactions on the majority of web apps are clicking around, going to different pages, listing stuff, etc.

Where StateChannel/LiveData/Storex comes to play, I think, is data mutations, especially with server-side validations. If there’s a form to add a record, edit it, maybe even collaboratively, or some sort of data dashboard as well that updates in real-time, we open a WebSocket connection to the backend, and we send messages to it and receive state updates that are synced in somewhat real time to the client, that updates in place. There’s also a possibility to do limited data loading this way, but if we were to base the whole application around it, we’d end up with a very similar set of issues you end up with when using LiveView: in particular reconnection problems, problems on slow/unreliable networks, no possibility to do anything offline, issues with handling big state as in very robust, fast-changing and interactive pages that need to sync a lot of data back and forth, etc.

I should probably put together some more advanced demo of this, I was looking for a more advanced “Todo MVC” app template to replicate and compare the approaches with, and I found Live Beats by @chrismccord https://livebeats.fly.dev/ which is very impressive. I plan to replicate the same thing with Next.js + StateChannel (i.e. just Phoenix Channels) and see which one I like more. This is all just experimenting for now.

Post #10

Load more posts (27 remaining)

Where Next?

View thread on forum (has 37 replies!)

phoenix

channels

react

Home Chat & Discussions>Discussions

56 5087 37

Last post

REPLY VIEW THREAD WITH 37 POSTS

Client-side rendering, server-side state (StateChannel)

hubertlepicki

Showing Posts 1 to 10 Show Best PostsShow All (oldest first)Show All (newest first)

kanishka

Sebb

Sebb

hubertlepicki

hubertlepicki

hubertlepicki

Sebb

methyl

Sebb

hubertlepicki

Where Next?

Trending in Discussions

What Elixir related stuff are you doing?

Hello Elixir World (Introductions thread)

My thoughts on Ash

What took you way too long to figure out?

Now that fly are pivoting, should I trust them for hosting?

What API mocking tool are Elixir teams using?

How do you pronounce ~> symbol?

Other Trending Topics

Hobbes - a low-level distributed database for the Elixir programming language

Beam Bots - Resilient Robotics on the BEAM

ExRatatui - Elixir bindings for the Rust ratatui terminal UI library

Emerge & Solve - a GUI framework for Elixir

Corex - Accessible and unstyled UI Phoenix components

Vutuv.de - an open-source LinkedIn alternative built in Phoenix

Latest Phoenix Threads

Chat & Discussions>Discussions

Latest on Elixir Forum

Sponsor Spotlight

Our Sponsors

Categories:

Sub Categories:

Forums

Popular Tags

Our Sponsors

We're in Beta

Options

Thread Display Mode

Thread Preview

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)