Corrupt transcoded video with rkmpp decoder.

[kaaku3]

The bug

X265 4K videos, transcoded down to 720p x265 using rkmmp hardware accelerated encoding and decoding are sometimes corrupted causing looped playback and strange colours and artifacts when played back.
Video seem fine with HW decoding disabled.
It does not happen every time and is inconsistent but happens often.

The OS that Immich Server is running on

Armbian Bookworm

Version of Immich Server

V1.118.2

Version of Immich Mobile App

V1.118.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

Same as default

Your .env content

Same as default

Reproduction steps

...

Relevant log output

No response

Additional information

No response

[kaaku3]

Hi, Are you sure this is fixed?
You mentioned in #13785 that HW decoding is untested.
The problem I reported only affects HW decoding.

[mertalev]

I tested this separately and I think it's caused by this issue. You're most likely seeing tone-mapping artifacts that will be fixed by setting the peak luminance explicitly. Since the linked PR does just this, it should fix the issue.

[kaaku3]

Thankyou very much for getting back to me, I appreciate all the hard work.

[kaaku3]

HI, Unfortunately I do still have the same problem on hardware decoded transcoded videos in v1.120.
It has significantly improved but is still happening in low light videos

[mertalev]

Hmm, I guess peak=100 (1000 nits) is off for certain videos. Can you share the output of mediainfo <path> or ffprobe -show_streams <path> for an affected video? Maybe we can use a different number based on certain metadata.

[mertalev]

Also, are the videos experiencing issues in 1.120.0 the same videos that looked corrupt in earlier versions? Or do different videos look corrupt now?

[kaaku3]

Media Info

General
Complete name : PXL_20241105_062315977.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 54.8 MiB
Duration : 7 s 250 ms
Overall bit rate : 63.4 Mb/s
Frame rate : 59.859 FPS
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC
xyz : +35.6247+139.4298/
com.android.manufacturer : Google
com.android.model : Pixel 6a

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L5.1@High
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 7 s 250 ms
Bit rate : 63.1 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Rotation : 90°
Frame rate mode : Variable
Frame rate : 59.859 FPS
Minimum frame rate : 59.367 FPS
Maximum frame rate : 60.362 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.127
Stream size : 54.5 MiB (100%)
Title : VideoHandle
Language : English
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 7 s 243 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 170 KiB (0%)
Title : SoundHandle
Language : English
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC

Other
Type : meta
Duration : 7 s 250 ms
Bit rate mode : Variable

[mertalev]

Hmm, it doesn't seem like this is an HDR video. Could you follow these steps:

1. Change immich's log level to debug in the admin settings under Log Settings
2. Run transcoding for this asset, making sure you have hardware decoding enabled
3. In the terminal, run docker logs immich_server
4. Find the latest log that shows an ffmpeg command (has /usr/bin/ffmpeg in it)

I'm curious to see what the specific command is to see where this is going wrong.

[kaaku3]

I set log level to debug, but unfortunately i do not see ffmpeg anything.
I tried verbose and the only thing I see that is remotely related in the log is:
[Nest] 17 - 11/09/2024, 1:18:31 PM VERBOSE [Api:LoggingInterceptor~p0aolibi] {"assetIds":[],"name":"transcode-video"}

[kaaku3]

Disregard that, the problem was the option "refresh encoded videos" is not re-encoding the video... maybe i misunderstood the meaning of that feature.

I made a new video in low light and uploaded it with hardware decoding on, the result was corruption towards the end of the file.

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

I then renamed the file and reuploaded it with hardware decoding off and the resulting file was fine.

ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

Media info for file

General
Complete name : Kirk/2024/2024-11-09/reupload.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 78.3 MiB
Duration : 10 s 515 ms
Overall bit rate : 62.5 Mb/s
Frame rate : 59.536 FPS
Movie name : Re-upload
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC
xyz : +35.6879+139.4576/
com.android.model : Pixel 7a
com.android.manufacturer : Google

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 10 s 515 ms
Bit rate : 62.2 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Rotation : 90°
Frame rate mode : Variable
Frame rate : 59.536 FPS
Minimum frame rate : 42.959 FPS
Maximum frame rate : 96.774 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.126
Stream size : 78.0 MiB (100%)
Language : English
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 10 s 505 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 246 KiB (0%)
Language : English
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC

Other
Type : meta
Duration : 10 s 515 ms
Bit rate mode : Variable

I think you are right they are not HDR, but with the latest update corruption only seems to happen in low light videos, where as before the update it seemed more random.

edit. Just ran a few tests and the artifacts and problems seem to happen when in low light but then a strong light source appears in the video, causing looped video, artifacts and strange colours.

[mertalev]

In the transcoding settings under Advanced, could you set the keyframe interval to 60? I wonder if that has an effect here.

[kaaku3]

Wow yes, its seems fine with "Maximum keyframe interval" set to 60, thank you so much.
Is it bad to keep it at 60 as opposed to auto?

[mertalev]

It will make the videos very slightly bigger, but otherwise no downside as far as I know.

[kaaku3]

Ok, thankyou.
keeping the file size down was the reason i'm transcoding at all, as my mobile data plan is slow sometimes.

Not sure if it's related but i just noticed a very very thin line at the top and bottom of the video that is not there when I use software decoding.

[mertalev]

The default in your case is 256, for reference. Can you try setting it to, say, 128? I'm thinking of changing the default for RKMPP to remedy this, but also trying to minimize the impact to file size.

[mertalev]

cc @nyanmisaka if you have any insight into what's causing this behavior

[kaaku3]

After deleting and re-uploading the same to files one a dark video and one bright.
with 60 still sometimes is causing an issue on both bright and dark video.
128 was very bad.
Haven't seen a problem with it set to 32 after a few uploads.
The line I mentioned only affects hardware decoded videos and only visible on the android mobile app, not on web.

[kaaku3]

Since software decode is working fine with rkmpp hardware encode on its defaults, is there not any settings to play with on the hardware decode side?
Just after some googling it seems not everyone is getting perfect results with ffmpeg and drm_prime, could that be the problem?
The hardware decoder or ffmpegs compatibility with drm_prime? I don't really understand any of this but is there an alternative to drm_prime with rkmpp?

[mertalev]

A max of 32 for a 60 FPS video is a keyframe roughly every 0.5s. This is okay, but I think it's the lowest that's reasonable to use.

What's particularly weird is that this is an encoding setting - it shouldn't be fixed by using software decoding. It's a combination of the frames being outputted by hardware decoding and the keyframe interval that's causing this issue.

I think drm_prime is needed for good performance so filters can directly access the frames in the GPU without copying them.

[kaaku3]

I'd like to help in anyway I can but I'm not very knowledgeable but i can follow instructions, do any of the devs have access to an rkmpp device?

[kaaku3]

I hope it doesn't seem like i'm complaining about the issue too much, I understand you must be busy on other things.
I love immich and i'm just grateful that rkmpp encode works at all, otherwise i'd have to buy a more power hungry device.

[nyanmisaka]

What’s the SoC model and rockchip linux kernel version? (uname -a)

Can u share a sample clip and the full ffmpeg command line?

I may have time to take a look tomorrow.

[mertalev]

No worries! It's great that you're reporting on this so we know about it and can fix it.

I think the best option would be to open an issue in jellyfin-ffmpeg as I believe most/all of the RKMPP decoders, filters and encoders for FFmpeg are implemented by them. A zipped sample video would be particularly helpful.

[mertalev]

According to OP, this command is problematic:

ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

This command works fine:

ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

And changing the first command to use -g 32 instead of -g 256 also works fine.

Also, it seems there's a line at the top and bottom of the video when hardware decoding that doesn't happen with software decoding. I think this happens with -g 32 as well, but I'm not sure.

[kaaku3]

uname -a
Linux immich-server 5.10.160-legacy-rk35xx #1 SMP Wed May 15 03:04:45 UTC 2024 aarch64 GNU/Linux
Its actually a rk3588s.
This is my sample video of a messy (excuse the mess) dark room that constantly causes issues.
2 files one with -g256 and one with -g 32.
In this instance -g 32 is also corrupt (up to this point -g32 was actually fine)
It seems hit or miss.

Thankyou for the command line noted above and the description is perfect, the line thing happens no matter the -g setting and is only visible in mobile app (android) maybe web viewer clips it off?

[w00tlarr]

Hiya - running into this issue too but with Intel iGPU hardware acceleration. No issues encoding without iGPU.

[w00tlarr]

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.

playback.mp4

[kaaku3]

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update.
The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot

[nyanmisaka]

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot

It's best to switch to a different client and try again.

[nyanmisaka]

For example, I cannot reproduce it on iPhone/iOS 17.

[kaaku3]

Ok, I will try it with the updated immich client, once the fix reaches it maybe it's not an issue anymore.
If it's still an issue I will open up an issue with the immich mobile app for android.

[w00tlarr]

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.
playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.

intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

[nyanmisaka]

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.
playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.
intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

I don't have an Intel GPU that supports SRIOV at the moment (Intel ARC A380), you could try looking in the Intel repo to provide some context. But anyway this issue is not related to the downstream software, as the same FFmpeg commands run perfectly on the host.

[mertalev]

@kaaku3 The bug fix for the RKMPP issue should be in 1.120.2. Can you try with that?

[kaaku3]

@mertalev that's great news, unfortunately my armbian server that immich was running went down last night.
I didn't touch anything, I don't know why.
You think publicly putting out a link to a video on immich was a bad idea?
I'm sure people have better things to do then mess with my server probably over thinking it.
Anyway I think I will need to put the immich backup restore feature to use today.
I will let you know if it's fixed as soon as i can.
Thankyou fire your help.

[mertalev]

Sorry to hear that! There are lots of reasons it could fail, like power loss. But in general, make sure the server is hardened with a reverse proxy, fail2ban, etc. if you're exposing it to the internet. Definitely don't expose Immich (or any other service) directly.

[kaaku3]

I was using nginx, I will look into fail2ban too thankyou.

[kaaku3]

Hi, I started fresh. I had a lot of problems seems the newer kernel don't fully support hardware acceleration... i'm not quite sure.
I went with Joshua Rieks ubuntu with legacy kernel, that apparent everything should work out of the box, but i had problems with opencl. It seems to be working now, but I'm not 100%. Is there anyway from immich log to tell if its falling back to software encode. It doesn't seem to be going as fast as it was before. Also immich restore didn't work for me so i'm uploading everything.

this is the ffmpeg from immich log

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/a7/f0/a7f09b0f-653d-4920-8a95-ec929b890cc8.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/f9/57/f9578312-e623-4a61-9667-607baed5a02e.mp4

anyway to tell from that that it isn't falling back to software?
I'm currently waiting on the problem file to be transcoded.

edit.

I there are errors

[Nest] 7 - 11/14/2024, 1:30:47 AM ERROR [Microservices:MediaRepository] handler_name : SoundHandle
vendor_id : [0][0][0][0]
Stream #0:20x3: Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, bt709, left), 3840x2160, 62337 kb/s, SAR 1:1 DAR 16:9, 59.86 fps, 59.94 tbr, 90k tbn (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : VideoHandle
vendor_id : [0][0][0][0]
[out#0/mp4 @ 0x3b53a170fc0] Adding streams from explicit maps...
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Created video stream from input stream 0:2
[hevc_rkmpp @ 0x3b53a2d1d80] Found SoC name from device-tree: 'rockchip,rk3588s-orangepi-5 rockchip,rk3588'
[hevc_rkmpp @ 0x3b53a2d1d80] Picked up an existing RKMPP hardware device
[aost#0:1/aac @ 0x3b53a1f0780] Created audio stream from input stream 0:1
Stream mapping:
Stream #0:2 -> #0:0 (hevc (hevc_rkmpp) -> hevc (hevc_rkmpp))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Starting thread...
[aost#0:1/aac @ 0x3b53a1f0780] Starting thread...
[vf#0:0 @ 0x3b53a127d40] Starting thread...
[af#0:1 @ 0x3b53a127e80] Starting thread...
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Starting thread...
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Starting thread...
Press [q] to stop, [?] for help
[hevc_rkmpp @ 0x3b53a2d1d80] Noticed an info change
[hevc_rkmpp @ 0x3b53a2d1d80] Configured with size: 3840x2160 | pix_fmt: drm_prime | sw_pix_fmt: nv12
[graph 0 input from stream 0:2 @ 0x3b542090300] w:3840 h:2160 pixfmt:drm_prime tb:1/90000 fr:60000/1001 sar:1/1 csp:bt709 range:pc
[Parsed_scale_rkrga_0 @ 0x3b542090240] w:3840 h:2160 fmt:nv12 -> w:1280 h:720 fmt:nv12
[graph 0 input from stream 0:2 @ 0x3b542090300] video frame properties congruent with link at pts_time: 0
[hevc_rkmpp @ 0x3b53a2d0500] Using input frames context (format drm_prime) with hevc_rkmpp encoder.
[hevc_rkmpp @ 0x3b53a2d0500] Rate Control mode is set to CQP
[hevc_rkmpp @ 0x3b53a2d0500] QP Init/Max/Min/Max_I/Min_I is set to 28/28/28/28/28
[hevc_rkmpp @ 0x3b53a2d0500] Tier is set to 1
[hevc_rkmpp @ 0x3b53a2d0500] Profile is set to MAIN
[hevc_rkmpp @ 0x3b53a2d0500] Level is set to 51
[hevc_rkmpp @ 0x3b53a2d0500] Configured with size: 1280x720 | pix_fmt: drm_prime | sw_pix_fmt: nv12
[graph_1_in_0:1 @ 0x3b540090300] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo
Output #0, mp4, to 'upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4':
Metadata:
major_brand : isom
minor_version : 131072
compatible_brands: isomiso2mp41
com.android.capture.fps: 60.000000
location : +35.6813+139.5357/
location-eng : +35.6813+139.5357/
com.android.manufacturer: Google
com.android.model: Pixel 6a
encoder : Lavf61.1.100
Stream #0:0(eng): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), drm_prime(pc, bt709, progressive, left), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 59.94 fps, 60k tbn (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : VideoHandle
vendor_id : [0][0][0][0]
encoder : Lavc61.3.100 hevc_rkmpp
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : SoundHandle
vendor_id : [0][0][0][0]
encoder : Lavc61.3.100 aac
[out#0/mp4 @ 0x3b53a170fc0] Starting thread...
frame= 138 fps=0.0 q=-0.0 size= 1024KiB time=00:00:02.02 bitrate=4133.1kbits/s speed=4.06x
[hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error submitting video frame to the encoder
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error encoding a frame: Generic error in an external library
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Task finished with error code: -542398533 (Generic error in an external library)
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Terminating thread with return code -542398533 (Generic error in an external library)
[vf#0:0 @ 0x3b53a127d40] All consumers returned EOF
[vf#0:0 @ 0x3b53a127d40] Terminating thread with return code 0 (success)
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Decoder returned EOF, finishing
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Terminating thread with return code 0 (success)
[vist#0:2/hevc @ 0x3b53a050d80] All consumers of this stream are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Terminating thread with return code 0 (success)
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder thread received EOF packet
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder returned EOF, finishing
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Terminating thread with return code 0 (success)
[af#0:1 @ 0x3b53a127e80] Filtergraph returned EOF, finishing
[af#0:1 @ 0x3b53a127e80] All consumers returned EOF
[aost#0:1/aac @ 0x3b53a1f0780] Encoder thread received EOF
[aost#0:1/aac @ 0x3b53a1f0780] Terminating thread with return code 0 (success)
[out#0/mp4 @ 0x3b53a170fc0] All streams finished
[out#0/mp4 @ 0x3b53a170fc0] Terminating thread with return code 0 (success)
[af#0:1 @ 0x3b53a127e80] Terminating thread with return code 0 (success)
[mp4 @ 0x3b53a190e00] Starting second pass: moving the moov atom to the beginning of the file
[AVIOContext @ 0x3b53a237fc0] Statistics: 2787923 bytes read, 0 seeks
[AVIOContext @ 0x3b53a237c00] Statistics: 5582216 bytes written, 4 seeks, 25 writeouts
[out#0/mp4 @ 0x3b53a170fc0] Output file #0 (upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4):
[out#0/mp4 @ 0x3b53a170fc0] Output stream #0:0 (video): 283 frames encoded; 278 packets muxed (2720898 bytes);
[out#0/mp4 @ 0x3b53a170fc0] Output stream #0:1 (audio): 192 frames encoded (196608 samples); 193 packets muxed (66981 bytes);
[out#0/mp4 @ 0x3b53a170fc0] Total: 471 packets (2787879 bytes) muxed
[out#0/mp4 @ 0x3b53a170fc0] video:2657KiB audio:65KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.230928%
frame= 278 fps=272 q=-0.0 Lsize= 2729KiB time=00:00:04.12 bitrate=5425.4kbits/s speed=4.04x
[aac @ 0x3b53a2d2480] Qavg: 832.837
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input file #0 (upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/40/f9/40f9b5f3-8512-4058-9ffe-b71c12358bac.mp4):
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:1 (audio): 193 packets read (99038 bytes); 192 frames decoded; 0 decode errors (196608 samples);
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:2 (video): 303 packets read (39917376 bytes); 289 frames decoded; 0 decode errors;
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Total: 496 packets (40016414 bytes) demuxed
[AVIOContext @ 0x3b53a230180] Statistics: 40437714 bytes read, 4 seeks
Conversion failed!

I don't think i can test this right now

[mertalev]

It seems like this is the relevant line:

[hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1

Does it fail for all videos, or only certain ones?

[kaaku3]

Thankyou for pointing that out,it seems to be only one file.
I was starting to think something was up with the drivers.
Looks as tho it was uploaded via an immich client on my wife's phone, is it possible it was something up with the upload...
It fell back to software transcode and passed that ok

[mertalev]

Thanks for clarifying. I'm assuming the corruption issue is resolved now?

I'm not sure what the issue is for that particular video, but if it worked with software decoding then it's most likely a bug.

[kaaku3]

Yes, I just transcoded that file and it's fixed, thankyou so much.
There is the line still but maybe that's a different issue.
I got a few more errors like the last
I will start again with my system make sure there is nothing up with the GPU drivers.
You wouldn't have any recommendations on a base image for orange pi 5 would you?

[mertalev]

ubuntu-rockchip tends to be recommended as the best for hardware acceleration support.

[kaaku3]

Ok, thankyou

[mertalev]

You can also try downgrading back to 1.120.1 to see if you still get the same error. That would make it clear whether the issue is due to the different server environment or the FFmpeg change.

[kaaku3]

I just wiped it ;), I'll keep that in mind if it persists

[kaaku3]

Sorry to keeps bothering you here... I think the base system is fine now, but in immich when i run transcoding despite having hardware decoding enable... I don't think it is.
here is the ffmepg output
ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/admin/2020/2020-10-24/PXL_20201024_021840973.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:1 -map 0:0 -g 256 -tag:v hvc1 -v verbose -vf scale=-2:720 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/a32baa7e-7326-459b-a8ab-f08034213bb7/14/de/14de9676-c1c3-4e73-80e8-e3b614773b6d.mp4

edit...

I just noticed this

[Nest] 7 - 11/15/2024, 11:39:53 AM DEBUG [Microservices:MediaService] OpenCL not available for transcoding, so RKMPP acceleration will use CPU decoding

So i think its just me and i need to get opencl working on my system

[mertalev]

Yes, the issue is the lack of OpenCL. There's a PR to loosen this restriction so it tries to still use hardware decoding and only do tone-mapping on CPU (if it's an HDR video).

The Failed to get key input frame from packet meta: -1 error you mentioned should be fixed in the next release.

[kaaku3]

Thankyou, yeah I was keeping an eye on jellyfin ffmpeg and I hoped that's what that commit meant.

edit...

I don't know how or why, but i after reinstalling everything opencl related, it is working now and hardware decoding is working too.

Thanks for all your help.

[kaaku3]

Hi I just wanted to add that I still have the same
Failed to get key input frame from packet meta: -1 in the latest update.

[mertalev]

Unfortunately the fixed version was merged two hours after the release :/

Maybe we can do a patch release.

[kaaku3]

Good to hear the fix is still on the way. Thankyou

[mertalev]

Can you confirm if the issue is fully fixed now on 1.122.1?

[kaaku3]

Hi, I'm currently away right now.
I clicked re-transcode all in web interface and kept an eye on it via SSH on my phone. For some reason my system suspended... Maybe I pocket typed something... I'm not sure.
I was able to check that it did about 4 thousand odd transcodes without error before it suspended though, so seems fixed.
Thankyou so much and thankyou for all the hard work.
I'm sorry I can't answer more definitely right now.

[mertalev]

Tentatively closing this issue since it sounds like it's fixed

[kaaku3]

I have rechecked my videos and I had absolutely no issues at all, including the line that appears in the video player, that appears fixed too.

Thankyou