jimmyhuang

jimmyhuang

How to do pagination in a nested graphql query with dataloader batch load?

I have a GraphQL server implemented with absinthe, dataloader and phoenix.

A sample query is as follows
What I anticipate is that the query lists the first 10 books in database and for each book, the first 10 chapters of the book.

{
  listBook(from:0, size:10){
    books {
      id
      name
      url
      chapters(from: 0, size: 10) {
        id
        name
        book{
          id
          name
          book {
            id
          }
        }
      }
    }
    total
  }
}

I am using dataloader to load chapters field.

However, the generated sql is not expected, instead of returning first 10 chapters for each book, the underlying sql actually returns first 10 chapters of all chapters of the first 10 books.

How can I achieve my goal? Make my GraphQL API return first 10 chapters of each book.

[debug] QUERY OK source="books" db=60.4ms
SELECT b0."id", b0."html", b0."info", b0."name", b0."preface", b0."text", b0."url", b0."info_html", b0."preface_html", b0."parent_id", b0."inserted_at", b0."updated_at", b0."parent_id" FROM "books" AS b0 WHERE (b0."parent_id" = ANY($1)) ORDER BY b0."parent_id" LIMIT $2 OFFSET $3 [[291289, 291066, 291009, 290794, 290647, 290409, 290196, 157109, 155130, 153693], 10, 0]

If I do not use dataloader but use a custom resolver, I can achieve my goal but that results in 1+10 queries because for each book the application needs to query database for its chapters.

Is there a way to use dataloader to batch load chapters?

The complete source code is here GitHub - huangjimmy/wikisource: A wikisource mirror site in Elixir and Phonenix · GitHub

Most Liked

jimmyhuang

jimmyhuang

I come up with a solution and it does work. I do use PostgreSQL window function. The solution will only work with PostgreSQL.

1 I add a virtual field chapter_number

lib/wikisource/book.ex

schema "books" do
    field :chapter_number, :integer, virtual: true

2 I inject a chapters: true parameter so that datasource.ex knows a query is a chapter query

lib/wikisource_web/graphql/schema/types.ex

    field :chapters, list_of(:book) do
      arg(:offset, non_null(:integer))
      arg(:first, non_null(:integer))
      resolve(dataloader(DataSource, :chapters, args: %{chapters: true}))

3 I construct the query with window function

I only select a subset of fields because some field is too large and costly to load.

lib/wikisource_web/graphql/datasource.ex

  def query(queryable, params) do
   case params do
      %{chapters: true, offset: offset, first: first} ->
        last = offset + first
        query = from r in queryable, select: r, select_merge: %{chapter_number: fragment("row_number() over (PARTITION by parent_id order by \"name\")")}
        from r in subquery(query), select: %Wikisource.Book{id: r.id, name: r.name, info: r.info, preface: r.preface, info_html: r.info_html, preface_html: r.preface_html}, where: r.chapter_number >= ^offset and r.chapter_number < ^last
      %{order_by: order_by, offset: from, first: size} -> from record in queryable, order_by: ^order_by, offset: ^from, limit: ^size

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New

We're in Beta

About us Mission Statement