Adding a subscription catchup function to Absinthe

suggestion
#1

This is a proposal mostly aimed at @benwilson512, but of course if anyone else has thoughts or suggestions I’d love to hear them.

So one of the limitations of GraphQL subscriptions, at least as implemented in Absinthe, is that there’s no way (that I can see) to start the subscription with a “catchup” set of data. For example, in an IM app, when I first connect I’d like to know who amongst my friends is connected. I’d also like to get live updates when any of them connect or disconnect. You can’t push data to the topic during the resolution, because the user only becomes subscribed to the topic after resolution has completed.

A couple of naieve approaches would be:

  • During resolution, spawn a process that will retrieve the data “later” - hopefully after the user is subscribed. That’s a terrible idea for a number of reasons I hope I don’t have to go into :slight_smile:

  • Have a separate “catchup” query which the client calls after the subscription. That works (and is what we’re using currently), but feels clunky and requires an extra query and resolution to be defined that would otherwise be unnecessary.

The addition of a continuation in the resolution system (as implemented in https://github.com/absinthe-graphql/absinthe/pull/559), though, gives us another option: Allowing the schema implementer to specify a “catchup function” that is called immediately after the initial subscription is set up but before the connection process is freed up to start processing live subscription data from other processes.

I’ve put together a prototype of this idea - you can see the changes at https://github.com/hippware/absinthe/commit/f748a60d49e1dccfb150535aa0c8b4609d80c821

From a schema implementer’s perspective, the change is pretty minimal: If you want to include catchup data, you just add a catchup function to the subscription definition:

    subscription do
      field :sub_with_catchup, :string do
         config fn args, _ ->
          {
            :ok,
            topic: args[:id],
            catchup: &Resolver.catchup/0
          }
        end
    end

When the continuation is called, it calls Resolver.catchup/0 and returns the data just as if it was a standard subscription event firing.

Does this sound like a reasonable idea? Is there some much better, more obvious existing solution I’ve missed?

Cheers,

Bernard

1 Like
#2

I don’t know a lot about graphql subscriptions, but I have a couple questions.

  • What is the common approach other people use to solve this problem?
  • Do other libraries that support subscriptions have a similar feature?

From my understanding of graphql, I would expect most people do the second approach you mention.

A problem that I could see with the catchup approach you mention is that you’ve removed the client’s ability to decide things about the catchup. With a separate query you have control over how much you catchup and how you catchup on the client. With the approach you’re suggesting, I believe every client would have to have the exact same catchup performed. If instead there were a way to incorporate the catchup query into the query to make the subscription, it would allow the client to continue to control what data it receives.

#3

Hey blatyo,

  • What is the common approach other people use to solve this problem?

I haven’t done an exhaustive review, but the RFC for GraphQL subscriptions explicitly allows for exactly this kind of thing: https://github.com/facebook/graphql/issues/283. The shortcoming, such as it is, is that Absinthe’s structure provides no way to implement this.

  • Do other libraries that support subscriptions have a similar feature?

Again, I can’t say I’ve looked closely, but my guess would be that it’s possible to implement in many libraries without them having to provide an explicit method for doing so. The limitation I’m seeking to overcome is that, currently, you simply can’t do it in a safe way in Absinthe.

A problem that I could see with the catchup approach you mention is that you’ve removed the client’s ability to decide things about the catchup.

Whether or not the catchup code was invoked could easily be controlled with an argument specified in the subscription. It could even be added to existing subscriptions, defaulting to “off” unless the client explicitly requested it. I don’t really see that being an issue.

1 Like