HW Accelerated encoding

[pptudela]

Has anyone successfully configured ffmpeg with Quick Sync or Nvenc to use hardware acceleration for encoding the different quality copies?

I presume it's just a matter changing the ffmpeg commands on tasks.py and adequately compiling ffmpeg, but was wondering if someone has done this before.

[DidacValenciano]

Was hoping to find an answer to that, and just find your question. It's not only matter of tasks.py but helpers.py too.

I'm trying some combinations. Without luck. Here's a sample of the logs:

logs/debug.log:FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg -hwaccel qsv'

It will need more effort ...

[Anduin2017]

Hi, any updates?

I bought an Nvidia Tesla P40. However, seems I couldn't benefit from it for running MediaCMS.

Any solution to use hardware accelerated encoding?

[Anduin2017]

For example, to encode a video to HEVC with CPU, we can run:

ffmpeg -i input.mp4 -preset slow -codec:a copy -codec:v libx265 -crf 20

But with GPU, we need to run:

ffmpeg -i input.mp4 -preset slow -codec:a copy -codec:v hevc_nvenc -rc:v vbr -cq:v 20 -rc-lookahead 10 -profile:v main10

According to my test, for CPU E5-2696v4 16 cores and Nvidia Tesla P40, using GPU, P40 has 5 times performance advantage.

[mgogoulos]

Hi @Anduin2017 , thanks for sharing this interesting info!

One would have to edit files/helpers.py and add the parameters manually for now to ffmpeg, since this is not possible through a web interface.
Cheers

[MyKeyTeeNZ]

Hi @mgogoulos - thanks for that, do you have an example of what we need to change in the helpers.py file for this to work? My GPU is Nvidia, just not sure where to edit at this point? Thanks in advance!

[mgogoulos]

have a look on files.helpers.produce_ffmpeg_commands, this is calling files.helpers.get_base_ffmpeg_commands , and is providing the ffmpeg command that is run for a single encoding (or commands, because there is a two pass encoding in some cases)

there's a lot of logic there and it's not easy to decouple, however it's a start! if you know what extra arguments ffmpeg is going to need, or need to adjust existing ones, that's the place to look

[dmurphynj]

helpers-vaapi.txt

Here is an updated helpers.py that I hacked FFmpeg vaapi support into.

I'm not a coder, don't play one on TV, nor am I an ffmpeg expert, but seems to be working ... haven't tested it much but it's a starting point.

Run a diff against the existing helpers.py and you can see what was changed.

[BadVolf]

Hey guys, here you go:

Rename it to helpers.py
helpers-hvenc.txt

This is working with my the NVidia RTX 3070IT Super. you will see it tests if the media should be hardware encoded and if so, it has filters for h.264 and h265

if use_hardware_encoding: encoder = "h264_nvenc" if codec == "h264" else "hevc_nvenc"

Sorry, I don't know anything about github, just thought I would share this with you.

Let me know if you have any improvements.

[HeneryH]

Your two configs have been a godsend, thanks mate! For anyone else trying to get hardware transcoding working, I found it easiest to replace ffmpeg with jellyfin-ffmpeg. Have had it working great on vaapi and nvenc using these two configs. It also helps to bump up the bitrates too, as software transcoding still looks a little better.

Glad it helps! I agree - using the jellyfin-ffmpeg package is a big, big help since it has everything compiled in.

I'm doing some interesting tricks here - my mediacms instance runs in a VM, under proxmox, on an Intel N100 CPU. It took quite a bit of work to get GPU passthrough to work, but I'm passing through the internal GPU in the N100 chip to the mediacms VM, so ffmpeg has direct access to it.

Complex config but working great - huge, huge improvement in transcode time vs. software-only.

The N100 CPU is ridiculously impressive for the cost... There's certainly better CPUs (and GPUs) out there, but for a 6-watt CPU? Insanely good.

@dmurphynj Dennis, I am trying to do the same and found the passthrough has been a challenge. Have you created any notes that describe your solution? I saw an interesting blog about passing through the GPU in what seems like an easy alternate method compared to the full GPU passthrough. I am in the middle of trying this Proxmox: Install Plex LXC with HW Transcoding/HDR Tone Mapping

[dmurphynj]

If you follow this to a T, it's what I'm using to make it work. Hasn't done wrong by me yet -- there's a LOT of different versions of this floating around, but Derek's Win11 version has been the most comprehensive and accurate thus far IMO.

I should also note that I'm not running any containers - this is a straight up VM.

https://www.derekseaman.com/2024/07/proxmox-ve-8-2-windows-11-vgpu-vt-d-passthrough-with-intel-alder-lake.html

edit: Added to say, of course you have to sub out Win11 for whichever Linux flavor you have -- and then install the DKMS i915 module inside that as well. I'm using Ubuntu, so the commands in my VM are something like:

sudo bash
cd ~
git clone https://github.com/strongtz/i915-sriov-dkms.git
apt install build-* dkms
cd ~/i915-sriov-dkms
dkms add .

dkms install -m i915-sriov-dkms -v $(cat VERSION) --force
dkms status

cp -a /etc/default/grub{,.bak}
sed -i '/^GRUB_CMDLINE_LINUX_DEFAULT/c\GRUB_CMDLINE_LINUX_DEFAULT="quiet i915.enable_guc=3"' /etc/default/grub
update-grub
update-initramfs -u -k all

[HeneryH]

Eh, I am running an 8th gen or so CPU. That method I think requires a 12th gen or newer.

[HeneryH]

I am using one of his other blogs about using the plex helper script that shows that a GPU can be passed in automatically. But I am not 100% sure of it. Proxmox: Install Plex LXC with HW Transcoding/HDR Tone Mapping

In this post I cover how to setup a Plex Media Server (PMS) LXC on Proxmox with hardware transcoding and HDR tone mapping. Contrary to popular belief, the Plex LXC does not need to be configured with PCIe passthrough for the GPU. So NO Proxmox IOMMU host configuration is needed to allow the Plex LXC to do hardware transcoding and HDR tone mapping. This procedure uses the awesome tteck Promox Plex LXC script to mostly automate the setup of a Plex LXC with the needed GPU drivers and LXC configuration to enable these GPU offload features.

[dmurphynj]

Eh, I am running an 8th gen or so CPU. That method I think requires a 12th gen or newer.

Frankly, the N100 based mini PCs are so cheap and effective, I'd consider replacing that 8th gen CPU with it. Shockingly good - I ditched an AMD 5950X rack mount server with a Beelink N100 minipc and the power savings alone are pretty close to paying for it. :-)

[Bewinxed]

How to make this work in the docker-compose image?

[mrjoshuap]

I've successfully added hardware encoding for Rockchip RK3588S arm64's, which is similar in modifications. I was however interested in thoughts on how to abstract the encoding profiles to make it a little easier to support different hardware encoders.

For my purposes, I had a custom build version of ffmpeg, bento4, and pass through transcoding hardware into the worker containers. If you're using a container deployment, make sure you pass through the devices into your docker-compose file. For instance, on RK3588s, you have to add the following devices to celery_worker:

  celery_worker:
    devices:
      - "/dev/dri:/dev/dri:rwm"
      - "/dev/dma_heap:/dev/dma_heap:rwm"
      - "/dev/rga:/dev/rga:rwm"
      - "/dev/mpp_service:/dev/mpp_service:rwm"
      - "/dev/video-dec0:/dev/video-dec0:rwm"
      - "/dev/video-enc0:/dev/video-enc0:rwm"

If you're using Nvidia or other hardware encoders, the devices will be different but will probably still be required...

It's also worth mentioning that when you are testing encoding profiles, grabbing the exact ffmpeg command being run is extremely important because a permissions problem on your devices usually ends up with a Segmentation Fault and basically no usable error messages in any of the logs.

[HeneryH]

Sorry if this is old or stale. Trying to get my intel embedded quicksync gpu that is being passed through Proxmox to work.

I see this in my helpers.py:

    if codec == "h264":
        encoder = "libx264"
        # ext = "mp4"
    elif codec in ["h265", "hevc"]:
        encoder = "libx265"
        # ext = "mp4"
    elif codec == "vp9":
        encoder = "libvpx-vp9"
        # ext = "webm"
    else:
        return False
 
   cmds = []
    for pass_number in passes:
        cmds.append(
            get_base_ffmpeg_command(
                media_file,
                output_file=output_filename,
                has_audio=media_info.get("has_audio"),
                codec=codec,
                encoder=encoder,
                audio_encoder=AUDIO_ENCODERS[codec],
                target_fps=target_fps,
                interlaced=interlaced,
                target_height=resolution,
                target_rate=target_rate,
                target_rate_audio=AUDIO_BITRATES[codec],
                pass_file=pass_file,
                pass_number=pass_number,
                enc_type=enc_type,
                chunk=chunk,
            )
        )
    return cmds

And I see this from my google search on how to do it...

FFmpeg Intel Acceleration

To enable Intel hardware acceleration for FFmpeg in MediaCMS, you need to configure FFmpeg to use the Intel Quick Sync Video (QSV) hardware acceleration. Here are the steps and considerations:

• Intel Quick Sync Video (QSV): Intel's hardware acceleration technology for video encoding and decoding. To use QSV with FFmpeg, you need to compile FFmpeg with support for libmfx (Intel Media SDK). Ensure your system has the necessary drivers and libraries installed.
• MediaCMS Configuration: For MediaCMS, you will need to modify the helpers.py file to include the QSV parameters manually since the web interface does not support this configuration. This involves adding the appropriate FFmpeg command parameters for QSV.
• Example FFmpeg Command: An example command for encoding a video using QSV would be:
  ffmpeg -i input.mp4 -c:v h264_qsv -preset slow -c:a copy output.mp4
ffmpeg -i input.mp4 -c:v h265_qsv -preset slow -c:a copy -global_quality 20 -b:v 0 output.mp4
  This command uses the h264_qsv codec for video encoding with the slow preset to ensure good quality.
• Performance Considerations: Intel QSV can significantly improve encoding and decoding speed compared to software-only processing. However, the performance gain depends on the specific Intel CPU and its support for QSV.
• Testing and Debugging: After making changes, test the configuration thoroughly to ensure that hardware acceleration is working correctly. Check logs for any errors or warnings related to FFmpeg or hardware acceleration.

Trying to figure out the proper way to inject the
c:v h264_qsv -preset slow
or
c:v h265_qsv -preset slow

[HeneryH]

Hey @dmurphynj , I am looking at your diff for the helpers.py file. Can you tell me what the intent of this change was? Not sure if I might be comparing and older version.

Comparing files helpers.py and HELPERS-VAAPI.TXT
***** helpers.py
stdout = run_command(cmd).get("out")
stream_size = sum([int(line.replace("|", "")) for line in stdout.split("\n") if line != ""])
video_bitrate = round((stream_size * 8 / 1024.0) / video_duration, 2)
***** HELPERS-VAAPI.TXT
stdout = run_command(cmd).get("out")
stream_size = sum([int(line) for line in stdout.split("\n") if line != ""])
video_bitrate = round((stream_size * 8 / 1024.0) / video_duration, 2)

---

[dmurphynj]

That looks like two different versions of helpers.py possibly?

I'm looking at the original helpers.py I spawned the VAAPI version from, and I didn't modify that code.

It was as follows:

        stdout = run_command(cmd).get("out")
        stream_size = sum([int(line) for line in stdout.split("\n") if line != ""])
        video_bitrate = round((stream_size * 8 / 1024.0) / video_duration, 2)

Not sure when it changed, but must be after the version I'm using.

[HeneryH]

Thank you. I figured. All the other diffs made sense to me.

Have you found a way to check if the GPU is running on transcode? I tried using intel-gpu-top on the proxmox host but It never shows any activity. And the CPU goes to 200% cpu with ffmpeg.

[dmurphynj]

Yeah, I use 'vainfo' to make sure all the profiles are available ...

And then I double check ffmpeg to make sure it's actually using the hardware .... something like this:

 time /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264 -I source.mp4 test_noaccel.mp4
 time /usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel qsv -c:v h264_qsv -i source.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=108' -c:v h264_qsv test_accel.mp4

Very primitive but this is what happens when I hack something in at 2am!