-
-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ log all errored requests #30
Conversation
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @joerunde! only minor comments.
It feels like it would be better for the response logging to happen in one place (whether success or error), not sure how realistic that is but in any case can save for possible later refactor.
Signed-off-by: Joe Runde <[email protected]>
Thanks for the review @njhill, everything looks like it still works! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @joerunde!
This PR logs all errors during validation or generation for a request like TGIS does. Signed-off-by: Joe Runde <[email protected]>
For each request is checked if the prompt length exceeds the largest target padding length as specified for the supported shapes. It that happens to be true, then the request is ignored and a corresponding message is output by the server.
Drafting some changes to log all errors during validation or generation for a request like TGIS does. The goal is to get something like this:
though probably without the file name/line number stuff (
text_generation_router::grpc_server: src/grpc_server.rs:429
in the example above)