Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Authentication Error #1062

Merged
merged 3 commits into from
Sep 20, 2023
Merged

Conversation

TristanCacqueray
Copy link
Contributor

@TristanCacqueray TristanCacqueray commented Sep 19, 2023

This change avoids retrying requests that fail with a 401 error. #1060

@TristanCacqueray
Copy link
Contributor Author

@morucci I've replaced the streaming error handling with a simpler approach in this PR. Though I don't expect it to fix the issue, and if it does, it would be good to know why. What we observed is that for take datas, the processing ends correctly:

2023-09-19 17:07:53 WARNING Lentille.GraphQL:235: Could not fetch the current rate limit {"index":"monocle2","crawler":"gc-monocle","stream":"TaskDatas"}
2023-09-19 17:07:53 WARNING Macroscope.Worker:132: Error occured when consuming the document stream ...

But for projects and pull-requests, the error is somehow ignored:

2023-09-19 17:08:03 WARNING Lentille.GraphQL:235: Could not fetch the current rate limit {"index":"monocle2","crawler":"gc-monocle","stream":"Changes"}
2023-09-19 17:08:03 INFO    Macroscope.Worker:188: Continuing on next entity {"index":"monocle2","crawler":"gc-monocle","stream":"Changes"}

@TristanCacqueray TristanCacqueray force-pushed the auth-error branch 4 times, most recently from 5db727c to 9ce14a6 Compare September 19, 2023 20:32
This change avoids retrying requests that fail with a 401 error.
This change simplifies the stream handling by:
- Using an explicit return value instead of a throw exception,
- Handwritten process batch stream consumer.
@TristanCacqueray
Copy link
Contributor Author

@morucci in the last commit I've also added a test to validate the logic, and the test pass with or without the refactor, so I think the issue is somewhere else in the GraphQL module.

@morucci
Copy link
Collaborator

morucci commented Sep 20, 2023

Thanks for the refactor. Let's merge it. However yes it is a weird that processStream do not receive the GraphError from streamFetch.

@morucci morucci added the merge me Trigger the merge process label Sep 20, 2023
@mergify mergify bot merged commit 818f553 into change-metrics:master Sep 20, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge me Trigger the merge process
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants