GitHub API rate limit

[tegioz]

With the actual number of repositories and teams, a validation job takes ~20 API requests, whereas a reconciliation one takes ~500 requests. The installation access token we'll be using for the CNCF org (enterprise cloud) has a rate limit of 15000 requests per hour, which means that at the moment we should be be able to handle a bit less than 30 pull requests per hour on the CNCF people repo that modify the config.yaml permissions file.

We have some ideas to both reduce the API usage and improve how the situation is handled should the rate limit be reached, but in the meantime we wanted to bring awareness about this limitation (although we don't expect it to be a problem in the short term).

/cc @jeefy @RobertKielty

[jeefy]

Could using something like https://github.com/kubernetes/test-infra/tree/master/ghproxy (or a new implementation of it) be helpful?

[tegioz]

Thanks @jeefy! I didn't know about ghproxy/ghcahe, added to the toolbox 😉

One of the ideas I mentioned we had to reduce API usage would be similar to that: the Rust GitHub library we use in CLOWarden (octorust) includes a feature named httpcache that does something similar transparently but within the same process (we just run a single instance of the CLOWarden server).

We still need to test this thoroughly to make sure everything works as expected, but we'll keep this in mind. Thanks again 🙂