How to set the Start time for Fast Indexing

[chrisrae75]

Hi,

Is there anyway I can set the time when fast indexing is performed?
Currently when I look at the logs, I can see the API for Fast Indexing is working, but it is performing the fast indexing
at the 30 minute mark of every hour - ie. 12:30pm, 1:30pm, 2:30pm
This then results in me losing the first 30 minutes of every live stream (mostly they start at the start of each hour).

Or, is there a way I can set the fast indexing to every 10 minutes? more frequency?
I dont want to set this up for every source, but I have a particular source that has a live stream and tears it down almost within 20 - 30 minutes after it has finished, so I dont have a lot of time to wait for indexing etc.

Many thanks in advance.

Chris
PS. I am new to Pinchflat, but so appreciative of this work.

[kieraneglin]

Hey there! Thanks for the report (:

The fast index frequency is actually set to run every 10 minutes. That said, I know this isn't what you've asked, but this question just gave me a big hint as to what could be happening in #395. I probably won't have time to dig in until early next week, but I'll see what I can find!

I have a particular source that has a live stream and tears it down almost within 20 - 30 minutes after it has finished, so I dont have a lot of time to wait for indexing etc.

This might be tricky to handle even if I get this other issue sorted just with how YouTube takes its time processing the livestream behind-the-scenes. I'm hoping that a solution to #395 would also fix this but until I dig in I can't say anything definitive

[chrisrae75]

Thanks mate!
No problems at looking at it next week.

Would be great to resolve :)

[chrisrae75]

Just had a look at #395 and from my side, grabbing during the stream is okay, but if there was a way to check how long the stream is (in hours:minutes) at the end of the live stream download, and compare immediately after the stream has completed. Then if there is a discrepancy in what Pinchflat downloaded, it could immediately initiate a redownload, that might help solve the problem. ie. If I missed the first 30 minutes of a live stream, so I only saved 1hr:30mins, and Pinchflat checks after the live stream has finished and see's it should have downloaded 2hr:0mins, it could issue a redownload command.
Just an idea :)

[kieraneglin]

The issue with grabbing during the stream is that the app only has two concurrent download workers, so it becomes super inefficient if one gets tied up waiting for a livestream to finish. I can't increase the number of workers because that increases the chances of people getting IP banned from YouTube

I think I have something workable in #485! It won't download a livestream until it's completed and finished processing. If you have fast indexing running, that should attempt re-download every 10 minutes which hopefully should be frequent enough for your needs 🤞

If you find it's still hit-and-miss after these changes, consider checking out these docs

[chrisrae75]

Thanks Kieran!Much appreciated for the i sight.I will have a read and give it a go.Another thing I noticed, is Pinchflat had started downloading a love stream, and not sure why, but for some reason after 5 hours it ended the download even though the love stream was still going.I thought I would redownload using “force download”, but it seemed not to download from the stsrt of the stream, but only from when I initiated the “force redownload”. So I lost the inital 5 hours and was left with only the last hour.Can the downloader start from the beginning of a live stream?CheersChrisOn 27 Nov 2024, at 3:56 am, Kieran ***@***.***> wrote: Closed #480 as completed via #485. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>