How to batch load with Absinthe Relay while supporting first: x and after_cursor?

Questions / Help
, , ,
1

I’m trying to use Absinthe Relay to create Relay compatible Graphql endpoint. I am having troubles to create connections with batching.

Trying to query this, which per organization the two youngest users:

query ConnectionOrganizationListPaginationQuery([...]) {
  organizations(first: 10 etc....) {
    edges {
      node {
        name
        users(first: 2, orderBy: {age: ASC}) {
          edges {
            node {
              name
              age
            }
          [...Cursors and PageInfo...] 
}

The example below works:

In schema

  connection field :users, node_type: :user do
      arg(:search_for, list_of(:user_search), description: "List of search objects")
      arg(:order_by, list_of(:user_order), description: "List of order objects")
      arg(:filter_at, list_of(:user_filter), description: "List of filter objects")
      resolve fn
        pagination_args, %{source: organization} -> 
         args = Map.merge(pagination_args, %{filter_at: [%{organization_id: ["#{organization.id}"]}]})
         UsersResolver.list(args, %{})
      end
    end

In resolver

  def list(args, _) do
     My.App.Users.Repository.list(args, %{role: :stub_admin})
    |> Connection.from_query(&Repo.all/1, args)
  end

This leads to the infamous N=1 problem. Using field :users, list_of(:user), resolve: dataloader(My.App.Users) works for batching, but doesn’t seem to be compatible with Absinthe Relay. I just can’t find the right way to do batch loading compatible with Absinthe Relay. Is it possible at all? If so, please provide a few hints or an example :slight_smile:

ps. I rather not load the whole list of all users in memory for pagination.
pps. Yes, I have searched myself :wink:

image
image.jpg1376×74 11.5 KB

2

The short version is: I don’t know.

The longer version is that if you look at this from a pure SQL perspective, the problem we’re trying to solve is this: Given a set of posts, how do you load the last 10 comments from each post. This is possible to do, but it requires either lateral joins or window functions, neither of which are easily supported by Ecto.

1 Like
3

Thank you Ben, I already took the lateral join path. Feels much better to know it’s hard and it’s not just me writing unnecessary complex code.

I will post the solution here…if I find one.

1 Like
4

Yeah I mean the issue with lateral joins is that while they’re sort of possible with Ecto, they require writing out the lateral join as a string, which makes it very difficult to do in a generic way inside a library.

5

For a comprehensive breakdown of how difficult this is see: How to address the issue with limits on preloaded associations using Ecto?

1 Like
6

Had a weekend without a screen (almost, it’s 23.18) but will have a good look at it tomorrow :slight_smile:

Have a working ‘first: x’ implementation (which indeed is not generic) but have to resolve ordering in the lateral join (using ecto, pure postgres returns in correct order).

@benwilson512 Why support Relay while this ain’t much of a possibility? Answer might be in you book but couldn’t read any further today (ereader doesnt count as a screen (a))

7

I’m not sure I understand this. Absinthe Relay encompasses a variety of functionality, the fact that Ecto makes it hard to do efficient connections inside connections, itself a relatively uncommon access pattern, doesn’t negate all of that.

1 Like
8

Sorry, was to tired to actually grab my phone to remove the comment once I realized I was 100% off with my question. Although your answer raises another question as you say connections inside connections, itself a relatively uncommon access pattern. Why is this relatively uncommon (googled, didn’t find a thing)? In your book you give examples of the advantage of single Graphql call over multiple REST calls. As far I can see right now, doing connections in connections is the way to reach the single call. For example:

recipes >- ingredients -> ingredients >- in_stock -> stores >- addresses -> addresses

Of course I could split those, and query for this information ‘just in time’ while navigating in the (web)app. But that seems to undo a little bit of the purpose of Graphql.

9

Hey @benwilson512 I’m curious if this story has changed any in the last 4 years? IIUC Ecto support for lateral joins and window queries has improved quite a bit and I’m presently looking at a project where I might be able to leverage AbsintheRelay.

1 Like