Tips for throttling requests and avoiding hitting the rate limit

[zeke]

Hi, @philschatz. This library is awesome! Thanks for keeping it up to date all this time. 🙏

Here's how I'm currently doing rate limiting:

const RateLimiter = require('limiter').RateLimiter
const limiter = new RateLimiter(5000, 'hour')
const Octokat = require('octokat')
const octo = new Octokat({token: process.env.GITHUB_ACCESS_TOKEN})

repos.forEach(repo => {
  limiter.removeTokens(1, function () {
    octo(repo.owner.login, repo.name).contributors.fetch()
      .then(contributors => // do stuff )
  })
})

This is working okay, but I wonder if there's a better way. Thoughts?

Also, I saw this in the README:

Listeners for rate limit changes

What exactly does that mean?

[philschatz]

Unfortunately, octokat does not perform throttling but it does provide the following which may be useful:

• eTag caching and you can provide your own cache handler to save to disk
• an "unofficial" listener that tells you the rate limit remaining and when the limit resets

To use the listener you can doing something like the following:

const Octokat = require('./dist/node/octokat')

const octo = new Octokat({
  emitter: function(eventType, details, response, rateLimit) {
    const {method, path, data, options} = details
    const {isBase64, isRaw, isBoolean, contentType} = options
    // The following are in seconds since epoch, not milliseconds
    const {remaining, limit, reset} = rateLimit
    console.log(arguments)
  }
})

octo.repos('philschatz/octokat.js').fetch()

// alternative:
octo.fromUrl('/repos/philschatz/octokat.js').fetch()

It really should be on the octokat object but I was not sure if:

1. people would find it useful
2. how much detail should be sent
3. whether Rate limits should be left as seconds since epoch or converted to Date objects

... but now that there are 2 requests for it, I'll add it 😄

I hope that helps!

Also, any chance electron-userland/devtron#87 (comment) might end up on the blog?

... I also sent an email to [email protected]