Syncing data from the server to the client (and vice versa) is a common problem to many applications. Although from what I can tell there aren’t any libraries or patterns that address this need for Phoenix Channels (I took a look here, hex.pm, and npm).
I want to avoid having completely bespoke syncing code on both the client and server. Right now the channel that I’m working on has operations like add_element, update_element, delete_element (for multiple values of element) and both the client and server have to create/handle those messages for each data type.
My target client-side is React, although the syncing code doesn’t need to be React-specific.
Does anyone know of any Phoenix-specific libraries or patterns for this?
Edit: just realized you want to go client to server as well. I would probably use standard Channels in that case. You could maybe use macros or something to extract common logic.
Yeah I’m looking for something bidirectional so PushEx doesn’t quite match.
Another way to think about what I’m talking about is an api like Phoenix Presence, but that is able to handle larger amounts of data (the CRDT implementation of Phoenix Presence gets slow if you try to put too much data inside it).
Which is inspired a bit by LV sending HTML diffs - instead of shipping entire JSON payload, why not json diffs?
This is not bidirectional - client sends actions/mutations - and the server responds with a JSON diff that can patch the client state accordingly.
IMO the complexity introduced with a CRDT is not worth for simply syncing data between client <> server - you are probably better served with a GQL style model with subscriptions and a client side cache for ‘synchronized’ data.
If for example some of the data needs to remain private - might not be easy to ‘exclude’ that from the CRDT, I am not well versed enough in CRDTs though so I could be wrong…
Can you tell us a little bit more about your scenario? Is it a ‘client synching with server’ or a ‘client synching with many other clients’ kind of situation?
I am working on a collaborative tool, but for now I’m okay with utilizing the server to serialize all the edits. But in the future I may want to look into a custom CRDT implementation.
I think that’s exactly the project that I am looking for! Although I’d have to look into the approach used to understand if it would be a good fit.
Yes, I don’t need bidirectional since I want to enforce some business logic on the server anyway. A GQL style model with subscriptions would be great, but I don’t want to go full GQL just because I want data syncing automatically (in a format that’s easily hooked up to React).
I would say it is multiple clients syncing with each other, but the server is acting as an intermediary between them (and serializing all of the operations).
It can be easy to hook something up to react - but optimizing it to ensure it does not result in full react tree re-renders is the tricky bit.
You’ll probably wanna lean on some react global state manager like redux or similar and hope your server-> client patches/updates can be precisely applied to the store and only trigger re-renders where necessary.
Might want to use a normalized structure like redux suggests to help make that easier - Normalizing State Shape | Redux (redux or not)
For json patches - might wanna try this newer library - jsonpatch | Hex - which is the most recently updated elixir rfc6902 (ietf.org) implementation
