Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up logging and use some technique to track logs for individual concurrent requests #122

Open
MRuecklCC opened this issue Jun 22, 2022 · 1 comment
Labels
quality Issues related to Code Quality and Complexity reduction

Comments

@MRuecklCC
Copy link
Contributor

MRuecklCC commented Jun 22, 2022

Use an approach as described here to improve logging: https://medium.com/gradiant-talks/identifying-fastapi-requests-in-logs-bac3284a6aa

Also we currently seem to have different logging settings going on at the same time:

extractor_1   | 2022-06-22 12:09:52,109 INFO metalookup.core.metadata_manager Extracted LicenceExtractor in  1.70s.
extractor_1   | 2022-06-22 12:09:52,155 INFO metalookup.features.adblock_based Found 1 links that should be blocked according to ad-block rules in  0.55s
extractor_1   | 2022-06-22 12:09:52,155 INFO metalookup.core.metadata_manager Extracted FanboyAnnoyance in  2.27s.
extractor_1   | 2022-06-22 12:09:52,328 INFO metalookup.features.adblock_based Found 0 links that should be blocked according to ad-block rules in  0.26s
extractor_1   | 2022-06-22 12:09:52,328 INFO metalookup.core.metadata_manager Extracted EasylistAdult in  2.44s.
extractor_1   | 2022-06-22 12:09:52,588 INFO metalookup.core.metadata_manager Extracted Accessibility in 12.92s.
extractor_1   | INFO:     127.0.0.1:54056 - "POST /extract?extra=true HTTP/1.1" 200 OK
extractor_1   | 2022-06-22 12:09:52,592 INFO root Warming up https://www.bpb.de/geschichte/zeitgeschichte/geschichte-im-fluss/142042/der-geopferte-rhein with task 5
extractor_1   | 2022-06-22 12:09:52,600 INFO metalookup.app.api Received request for https://www.bpb.de/geschichte/zeitgeschichte/geschichte-im-fluss/142042/der-geopferte-rhein
extractor_1   | 2022-06-22 12:09:54,239 INFO metalookup.features.adblock_based Found 0 links that should be blocked according to ad-block rules in   2.6s
extractor_1   | 2022-06-22 12:09:54,239 INFO metalookup.core.metadata_manager Extracted Advertisement in  4.35s.
extractor_1   | 2022-06-22 12:09:57,986 ERROR metalookup.core.metadata_manager Failed to extract accessibility

The subprocess for cache warmup probably uses yet another logging configuration.

@MRuecklCC MRuecklCC added the quality Issues related to Code Quality and Complexity reduction label Jun 22, 2022
@MRuecklCC
Copy link
Contributor Author

Alternative howto: https://github.com/snok/asgi-correlation-id

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quality Issues related to Code Quality and Complexity reduction
Projects
None yet
Development

No branches or pull requests

1 participant