Fixed-Function Media Engines in GPUs with HW Media Acceleration Features

[softworkz]

Intel

Intel GPUs have fixed-function media engines for quite a while. The important parts to know about are:

• VDBOX: Video Decoding
• VEBOX: Video Enhancement
• SFC: Scaling and format conversion
• VDEnc: Video Encoding

SFC was introduced in Kaby Lake to achieve low-power video playback without using shaders for scaling via EUs (Execution Units). EUs are the primary workhorses which are used by games for 3D computations but also for desktop composition, overlay and scaling of gpu surfaces. This is also what accounts for "3D" usage percentage in task manager.

Some video processing features are provided as EU Kernels, some via VEBOX fixed function blocks, sometimes, both. It depends on the GPU generation. Scaling is done via SFC since Kaby Lake

References:

• https://www.anandtech.com/show/9562/intels-skylake-gpu-analyzing-the-media-capabilities
• https://www.x.org/docs/intel/LKF/intel-gfx-prm-osrc-lkf-vol11-mediaengines.pdf
• https://www.x.org/docs/intel/KBL/intel-gfx-prm-osrc-kbl-vol15-sfc.pdf
• https://github.com/intel/media-driver#components-and-features
• https://github.com/intel/media-driver/blob/master/docs/media_features.md#supported-video-processing-feature-combination

SFC is not a generic and independent feature. It is part of the Media Engines and requires a media pipeline context to be used. SFC cannot be used for generic scaling, such as is done for desktop composition. For desktop composition, scaling is done by EU kernels.

ffmpeg Implementation

The QSV processing filter in ffmpeg has a scale_mode option which allows you to switch between shader-based (EU kernel) scaling and fixed-function block (SFC, also called 'low-power mode') scaling:

With EU Kernel Scaling

ffmpeg -hwaccel qsv -c:v h264_qsv -i video.mkv -vf "vpp_qsv=w=1920:h=1080:scale_mode=2" -f null -

ffmpeg speed=7.69x
CPU load: 25%

With Fixed-Function Scaling

ffmpeg -hwaccel qsv -c:v h264_qsv -i video.mkv -vf "vpp_qsv=w=1920:h=1080:scale_mode=1" -f null -

ffmpeg speed=8.05x
CPU load: 25%

CPU Decoding and Scaling

ffmpeg -i  video.mkv -vf "scale=w=1920:h=1080" -f null -

speed=8.03x
CPU load: 100%

NOTE: The test file I used is "Samsung Dubai" which you can find on DemoLandia.net

[softworkz]

DirectX VideoProcessor Scaling

Just few weeks ago I have added a new "Power Efficient" playback mode to our Windows app which is using mpv. MPV has its own presentation-side scaling implemented by custom shaders, which may provide good scaling quality but are also draining batteries quickly.

Our Win App with shader-based presentation scaling

Our Win App with D3D11 VideoProcessor Scaling

The presentation-layer scaling is disabled in this case

Windows Media Player

For comparison

---

Test file parameters: Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x1600 [SAR 1:1 DAR 12:5], 24 fps, 24 tbr, 1k tbn (default)
Playback scenario: 4k TV with HDR enabled, but playing in a window of about half the size.

[softworkz]

In that mpv based app, I also have an additional mode:

HW Prescaling means that D3D11 VideProcesor scaling is only applied when the video is significantly larger in resolution than the display on which the app window resides. Like 4k video on a laptop. These are often not capable to handle 4k frame downscaling via shaders (or at least pushing the GPU's 3D load very hard).

In that case it applies D3D11 VideoProcessor scaling but with a target size that is still somewhat larger than the window. This reduces the amount of data for the eventual scaling that is done for presentation via shaders (and also adapts to window resizing while the D3D11 VideoProcessor scale factor remains unchanged.

Side note: double downscaling (bilinear or bicubic) doesn't produce artifacts