-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Debug flaky CI integration tests #127
Conversation
Current theory: flakiness is caused by rate limiting. |
Should consolidate into a single test suite class.
1412b12
to
11e378e
Compare
Even with all these changes (slowing things waaay down) the build is flaky. I'm curious: can I up the parallelism again? |
0140a5e
to
591e490
Compare
591e490
to
3f7d1c2
Compare
This conclusion sucks, but I think I'm trying to adapt the tests to accommodate temporarily flaky API behavior. As I've poked around, the behavior has been unstable (e.g. earlier only HTTPS worked; now only HTTP works). It's possible this is driven by arXiv is tightening their rate limiter behavior... but that only explains this if they enforce the rate limit through a variety of unclear responses (TCP hangup vs. empty result set vs...). I might consider simplifying this PR to two fixes:
|
3e55127 might've just been lucky. |
ed2750f
to
49c3b03
Compare
Closing; PRs in the v2.0.0 release handle the best changes explored here. |
Description
CI tests are failing for latest commit on master. I'm unable to reproduce locally. Debugging.
Breaking changes
Relevant issues
Unexpected test failures:
Checklist
README.md
example usage has been updated.