Implemented hardware decoding

[matthewlai]

This implements hardware decoding continuing from the original work of @rvillalba-novetta and refactoring by @mikeboers in main...rvillalba-novetta:PyAV:hwaccel (and children commits)

I've completed the refactoring, fixed a few bugs, and brought the changes up to date with both PyAV at HEAD and ffmpeg 7.

I've also added a test and an example script. The test cannot be run in CI for obvious reasons (GitHub runners don't have GPUs), but I've manually run it on Mac, Windows, and Linux (WSL).

I understand that @mikeboers didn't find it worthwhile back in the days, but the situation has changed significantly since then - now almost every machine has a supported decoder, and modern encodings (HEVC, VP9, AV1) are so compute-intensive to decode that they have practically been designed for hardware decoding. As an example, I've been working with 4K HEVC videos (standard format coming out of action cameras and phones these days), and my Apple M3 can only decode them at 15 fps, while VideoToolbox easily does 100+ fps. I believe now and especially going forward, hardware decoding will be a necessity for almost any application that cares about performance or need realtime decoding.

Performance tests with a 4K HEVC video (using the new example script) -
MacBook Air M3:
CPU (M3) - 16.3 fps
GPU (M3, videotoolbox) - 121.4 fps

Windows:
CPU (AMD Ryzen 5 3500X) - 15.6 fps
GPU (NVIDIA RTX 3060 Ti, using d3d11va) - 49.1 fps

Linux (WSL):
CPU (AMD Ryzen 5 3500X) - 15.5 fps
GPU (NVIDIA RTX 3060 Ti, using cuda) - 91.3 fps

Notes:

• Windows accelerated performance is much lower than using ffmpeg directly (ffmpeg -hwaccel d3d11va -i ... -f null -). I don't know why but it's hard for me to work on Windows because I haven't figured out how to build PyAV on Windows and have to rely on GitHub Action. Using ffmpeg I get around 120 fps with d3d11va, and 150 fps with cuda. Maybe we need threading to interleave PCI-E transfers with decode?
• On Linux it's a bit slower but probably close enough. On Mac it's almost as fast as ffmpeg CLI.
• The ffmpeg included in wheels already has videotoolbox and d3d11va/d3d12va, so Windows and Mac are pretty well covered. Linux mostly only has vendor APIs and none are being built right now. Windows would also benefit from vendor APIs, but D3D11 is probably good enough for now (for NVIDIA it's about 15% slower).
• No new failures in existing tests.

Thanks!

[WyattBlue]

Why are you doing this?

[matthewlai]

This was @mikeboers 's change. I suppose because having two fields that are just opposites of each other is redundant?

[WyattBlue]

make lint is failing.

12 tests are also failing on my physical MacOS machine. The errors are the same as Github Actions

[matthewlai]

Sorry about that. I thought they were existing failures but I forgot to do a make clean when I syned back, and apparently the .so's didn't get rebuilt? I'll investigate, unless you can see why formats may be getting set to None in streams?

[matthewlai]

All tests pass locally for me now. Also made make lint happy.