MalloZup

MalloZup

Hi all,

I have developed in past an utils for maintaining opensource projects especially issues in GitHub.

As current limitation, the CLI can’t access more then 100 Issue. (GitHub - MalloZup/blacktango: console client for software maintainers · GitHub)

My question would be, do you know/ or have experience with the way for accessing all the issue in a project using Graphql?

Like if i would have 5000 issues, how would this work with Graphql GitHub

See my current query blacktango/lib/github/graphql.ex at master · MalloZup/blacktango · GitHub

Showing Posts 1 to 9

LostKobrakai

LostKobrakai

I’d imagine it’s a hard limit by GitHub to limit execution time of queries. If you need more use pagination to query the rest in addition requests.

Phillipp

Phillipp

The API supports pagination.

See Repositories - GitHub Docs

Scroll down to “issues”.

Phillipp

Phillipp

The problem with the GitHub GraphQL API is, that your rate limit is very low and they got pagination for everything. I once tried to fetch all issues/pull requests with all its comments and actions to analyze the data and give the repository a “rating” based on maintainer reaction times. It’s impossible if you want to stay inside the rate limit and the complexity of the queries is also huge since you would have to deal with multiple paginations at once if you want to use the least amount of queries.

MalloZup

MalloZup OP

Thx! If you have maybe some examples in Github would be really appreciated.. :heart_with_arrow:. E.g I’m pretty new to the graphql syntax and also their doc is kind complex​:grin:

Phillipp

Phillipp

Man are you lucky today. The GitHub GraphQL Explorer had one of my test queries from 2017 stored in my localstorage.

{
  repository (owner: "bolt", name: "bolt") {
    name
    issues (last: 5, before: "Y3Vyc29yOjIxMDI0NDc1NA==") {
      totalCount
      edges {cursor}
      nodes {
        number
        title
        comments (first: 5) {
          edges {cursor}
          nodes {
            body
          }
        }
      }
    }
  }
}

the edges {cursor} is important because they give you the IDs for the before and after params.
As you can see in the query, I get some issues and then its comments. But the comments are also limited to a specific number so you have pagination there too.
(and comments have reactions (paginated) and issues have actions (opened, closed, etc. ) (paginated too)). It really is a mess if you want to grab everything :smiley:

MalloZup

MalloZup OP

THx! I was hoping that there was a more clever way to have all the issue in a chunk instead of iterating over 100 and keep the Cursor id updating over them. :sob:

Phillipp

Phillipp

Yeah no, GitHub doesn’t really like it if you want to grab all the data. And since the rate limit is pretty low on the GraphQL API, you might even hit it while fetching all issues on a repository.

MalloZup

MalloZup OP

:sob: the V3 API in that sense for GH was imho at least better, for this use-case I never had issues like this.

Phillipp

Phillipp

You mean the REST API? Yeah, it is definitely easier to use but it still has pagination on everything. So while having a bigger rate limit, you also need to make more requests to the API to get the data you need.

— All posts loaded —

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
Onor.io
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
jaybe78
Hello, I’m developing a online persistent chat system (what’s app) like using elixir/dynamodb/aws for a mobile app(flutter). The diffic...
New
Trolleger
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
widianto
I think I’ve found a small improvement I could contribute to <%= web_namespace %>.CoreComponents (installer/templates/phx_web/compo...
New

Other Trending Topics Top

garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New
aseigo
ICal is a library for interacting with iCalendar data. It parses iCalendars into typed Elixir structs via ICal.from_ics, and can prepare ...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews