Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop calling yt-dlp three times in a row #357

Closed
benoit74 opened this issue Oct 8, 2024 · 2 comments
Closed

Stop calling yt-dlp three times in a row #357

benoit74 opened this issue Oct 8, 2024 · 2 comments

Comments

@benoit74
Copy link
Collaborator

benoit74 commented Oct 8, 2024

Currently, the scraper calls yt-dlp three times in a row for every video:

  • once to download the video (and audio)
  • once to download the thumbnail
  • once to download the subtitles

video and thumbnails are cached on S3 so the call to yt-dlp might not happen if we already have populated S3 cache for this video.

In the case of a new video where cache is empty, those three calls happens.

I think we should consider to refactor the code to call yt-dlp only once, because it will probably help avoid ban by Youtube.

@kelson42
Copy link
Contributor

kelson42 commented Oct 12, 2024

Very supportive of this ticket considering that hitting the quotas is a very serious limitation. Anything able to improve the statu-quo without removing features should be strongly considered. Pinning.

That said, @benoit74 is that not somehow a duplicate of #122?

@kelson42 kelson42 pinned this issue Oct 12, 2024
@kelson42 kelson42 modified the milestones: backlog, 3.3.0 Oct 12, 2024
@benoit74
Copy link
Collaborator Author

Indeed, this is a duplicate (but better written ^^)

@benoit74 benoit74 closed this as not planned Won't fix, can't repro, duplicate, stale Oct 14, 2024
@kelson42 kelson42 unpinned this issue Oct 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants