MalloZup

MalloZup

Hi all,

I have developed in past an utils for maintaining opensource projects especially issues in GitHub.

As current limitation, the CLI can’t access more then 100 Issue. (GitHub - MalloZup/blacktango: console client for software maintainers · GitHub)

My question would be, do you know/ or have experience with the way for accessing all the issue in a project using Graphql?

Like if i would have 5000 issues, how would this work with Graphql GitHub

See my current query blacktango/lib/github/graphql.ex at master · MalloZup/blacktango · GitHub

Showing Posts 1 to 9

LostKobrakai

LostKobrakai

I’d imagine it’s a hard limit by GitHub to limit execution time of queries. If you need more use pagination to query the rest in addition requests.

Phillipp

Phillipp

The API supports pagination.

See Repositories - GitHub Docs

Scroll down to “issues”.

Phillipp

Phillipp

The problem with the GitHub GraphQL API is, that your rate limit is very low and they got pagination for everything. I once tried to fetch all issues/pull requests with all its comments and actions to analyze the data and give the repository a “rating” based on maintainer reaction times. It’s impossible if you want to stay inside the rate limit and the complexity of the queries is also huge since you would have to deal with multiple paginations at once if you want to use the least amount of queries.

MalloZup

MalloZup OP

Thx! If you have maybe some examples in Github would be really appreciated.. :heart_with_arrow:. E.g I’m pretty new to the graphql syntax and also their doc is kind complex​:grin:

Phillipp

Phillipp

Man are you lucky today. The GitHub GraphQL Explorer had one of my test queries from 2017 stored in my localstorage.

{
  repository (owner: "bolt", name: "bolt") {
    name
    issues (last: 5, before: "Y3Vyc29yOjIxMDI0NDc1NA==") {
      totalCount
      edges {cursor}
      nodes {
        number
        title
        comments (first: 5) {
          edges {cursor}
          nodes {
            body
          }
        }
      }
    }
  }
}

the edges {cursor} is important because they give you the IDs for the before and after params.
As you can see in the query, I get some issues and then its comments. But the comments are also limited to a specific number so you have pagination there too.
(and comments have reactions (paginated) and issues have actions (opened, closed, etc. ) (paginated too)). It really is a mess if you want to grab everything :smiley:

MalloZup

MalloZup OP

THx! I was hoping that there was a more clever way to have all the issue in a chunk instead of iterating over 100 and keep the Cursor id updating over them. :sob:

Phillipp

Phillipp

Yeah no, GitHub doesn’t really like it if you want to grab all the data. And since the rate limit is pretty low on the GraphQL API, you might even hit it while fetching all issues on a repository.

MalloZup

MalloZup OP

:sob: the V3 API in that sense for GH was imho at least better, for this use-case I never had issues like this.

Phillipp

Phillipp

You mean the REST API? Yeah, it is definitely easier to use but it still has pagination on everything. So while having a bigger rate limit, you also need to make more requests to the API to get the data you need.

— All posts loaded —

Where Next? Top

Trending in Questions Top

katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
achenet
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Cxx-mlr
I’m working on a small exercise involving update_in/3, and I came up with this solution: data = %{ name: "Periodic Table", category:...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New

Other Trending Topics Top

budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews