Allow receiving an external frame index

[scotts]

🚀 The feature

We have had several request for the ability for VideoDecoder to accept an external frame index upon startup. That is, in exact mode, we build up a frame index which tells us the mapping between frame indices and pts. If users could provide that index to us, we could be in exact mode but not have to scan the entire file at startup. (We would, however, be the the mercy of the accuracy of the external index.)

Motivation, pitch

The main motivation is performance. There is potentially less need for this feature now that we have approximate mode (#427), which also avoids scanning the entire file. But the combination of receiving an external index and exact mode could perhaps yield the best of both worlds: exact seeks and fast startup.

The reason this comes up is that in many environments, videos are pre-processed before being used for training and inference. Creating and storing the external index for a particular video would be relatively low-cost, and it would be paid once.

[NicolasHug]

From a UX perspective, but also in terms of implementation, we may want to think of the user-provided index as a third seek_mode? The reason I say this is because when the index is provided by the user, we end up in a hybrid mode that doesn't really fit neither approximate nor exact. It's more of a combination of both:

• approximate: we don't scan, we don't rely on an index
• exact: we scan and rely on the index we built ourselves
• user-provided index: we don't scan (like in approx), but still use an index (like in exact).

[scotts]

That's a good point. I was thinking of the two concepts as being orthogonal, but you're right, they're related. That is, it doesn't make much sense to provide an external index if you're also using approximate mode - why provide us with something that you then ask us to not use? Ideally, our API would prevent such logical inconsistencies.