Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect retry behavior on v4/reports:batchGet #37

Open
glqstrauss opened this issue Nov 11, 2020 · 0 comments
Open

Incorrect retry behavior on v4/reports:batchGet #37

glqstrauss opened this issue Nov 11, 2020 · 0 comments

Comments

@glqstrauss
Copy link

glqstrauss commented Nov 11, 2020

With this tap now using v4 of the Google Analytics API, the retry behavior implemented by this tap is not longer consistent with recommendations in the GA v4 API reference.

  • This tap currently retries all 429s with backoff. The v4 documentation describes multiple cases for 429s, however, including one where the request should not be retried because a daily limit has been exceeded. We hit this a few times on a backfill, but it took a long time for the extraction to finally fail because the tap was backing off a daily quota 429.

  • 503 errors are never retried, but in the v4 API these errors are often transient, and the documentation suggests they should be retried with exponential backoff. Lack of retries here has proven to be particularly problematic when running the tap on a large GA report. 503s seem to be fairly common on requests to the batchGet endpoint that reach ~100 pages, and are recoverable, but no attempt is made to recover.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant