How to update a GitHub Repo daily

I have a GitHub project in which I store data collected by bicycle counting stations scattered around Cologne, Germany. At the moment, I update the data manually by running a Python script, which fetches the data and appends it to CSV-files. I then add and push the changes using git to the GitHub Project.

I would like to automate this process using an existing Elixir + Phoenix application I have running on Heroku. Since that app doesn’t do much, I thought that I could add a reoccurring job, maybe using Oban, which would fetch and push the data on a daily basis. The problem that I am facing now is how to implement this “in a smart way”.

My current plan is to clone the git repository to my Heroku machine and perform the git actions using e.g. elixir-git-cli. However, this feels a bit clunky and I was wondering whether you have an idea or experiences with automating reoccurring jobs which use git commands? Is there maybe a way to push changes to GitHub without having to clone a git repository first?

You can create commit via GitHub API directly.

Perhaps I’m misunderstanding the docs, but you would still need a git client to push the git object to GitHub’s remote to use that API, it doesn’t seem to offer a way to send a patch with your API request?

Rather than using the Elixir application could you run the code on GitHub actions? It supports periodic jobs. https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule

1 Like

You can create blob and tree. This will require some manual labour to properly define such commit, but you do not need Git client at all.

1 Like

Oh, that’s a great idea since I then don’t even have to re-purpose my other application in order to run this script! I’ll check it out, thank you :slight_smile:

Hmm, that sound’s like a possible, but complex solution. I’ll check whether I can use GitHub Actions first, but if not, then I’ll have a closer look to this solution. So, thank you :slight_smile:

Thank you for the explanation:)

I ended up using this GitHub Action to commit and push any changes made to the repo while running the GitHub Action. The full workflow can be found on GitHub. Thanks for the solution @lpil!

3 Likes

I wrote up my experiences in a short Medium post :slight_smile:

4 Likes