Subscription catchup / state delta

My solution is probably costly; here it goes: Do not embed the state in the notification at all.

For example, if a user comes on line or goes off-line, the same notification would be sent indicating that there is something happening wrt to the user. The subscribers of the notification have to issue a follow-up query to find out the state and compare the result to its own copy of the state.

That would be an option, certainly, but it seems like the worst of all worlds. It’s expensive (as you note) both computationally and in terms of network traffic; it still requires the client to have smarts that it ideally shouldn’t need (albeit slightly different ones), and it potentially produces redundant traffic when a few updates happen in quick succession (in which case the client’s first query may get all the updates at once, but it would still have to issue another query for each update notification to be certain).

For your example user presence it seems fine to me if you just

1. Subscribe for changes
2. Query for the initial state
3. Handle changes (including the ones before the query)

Worst case the user of the IM client sees a quick blip in a friends online indicator or gets a useless notification in the UI.

But i haven’t used Graphql so not sure if theres something i’m missing.
I have a similar use case but i’m using PostgreSQL logical replication which has an initial snapshot.

I am still a bit new to GraphQL, but one approach to model this might be to construct a subscription for the union of the ‘InitialState’ and ‘DeltaOfChanges’ types.

Because you then only make a single request (to establish the connection for the subscription), the issue of the race condition would be solved.

You’re quite right that that would in fact work for the example I gave. In retrospect, I should have given something slightly more complex, so replace it with a chat channel where you need:

a) The initial state (all messages up to this point), and
b) Notification of all changes to that state (when a new chat message is sent).

In this case you have to have the client de-duplicating chat messages that might have arrived on the subscription and also be in the state, and the Absinthe architecture doesn’t really make it easy to fix that. Again, in principle it’s easy enough to do on the client, but it feels like it shouldn’t be required.

That’s a neat idea (and I’ll probably use it at some point) but it doesn’t address the issue, which is that in Absinthe the subscription updates are handled asynchronously, so you can’t guarantee ordering/consistency between them and the initial state.

It would also require the un-merged “catchup” feature I linked earlier, but that, at least, is more or less a solved problem.

Thanks for your reply; you’re right, subscriptions are handled fully asynchronously which makes this more difficult. Essentially, we’d want to execute code whenever a new subscription is created, to send some data to only that subscription (rather than ‘everyone subscribed to this topic’ as is usually the case with Absynthe’s subscriptions).
Doing this kind of thing is very simple when e.g. using Phoenix Channels directly, but I’m not sure whether it is currently possible to access these internals from outside of Absinthe’s wrapper of them.

I wonder if @benwilson512 has some ideas on how to approach this situation.

If you really want to embed state delta in the notifications, then you should carefully design the delta format so they are idempotent.

You are describing a presentational issue here imo. If you include timestamps just let the client figure out the ordering and what to do?

This sort of thing is ideally implemented within Absinthe itself so that it isn’t directly dependent on the channels impl, but it’s a bit non trivial to sort out when to trigger the update. The challenge is that when you run a subscription, what is returned is a topic. The client then needs to subscribe to the topic, and this act of subscribing is just a call into the pubsub behavior, it isn’t something that pings any Absinthe code. That is to say, Absinthe has no way of knowing today when the end client actually subscribes.

Any proposed solution here would basically require then three things:

1. An Absinthe.Subscription.prime/n function that takes a client topic (as distinct from the general shared topic returned from config) that you want to prime.

2. Some way of figuring out what that root_value is. This could probably be a function returned from config on the subscription called maybe prime:. Eg: prime: fn -> Repo.get(Shipment, args.shipment_id) end).

3. Code in Absinthe.Phoenix to call prime.

A PR containing these two ideas would be welcome.