Hardware encoding seems to wipe off the timestamp

[SkyWriter]

Thank you for the wonderful product! I've spent hours fighting ffmpeg before I was able to find a turnkey go2rtc solution.

I am however experiencing some issues while trying to replicate my setup: have both hardware accelerated encoding and add a timestamp to the video. Performance is a non-issue for me, since I am transcoding a webcam stream into H264 anyway.

I am using go2rtc version 1.9.7 () linux/amd64.

Here's the setup that works:

streams:
  linux_usbcam: ffmpeg:device?input_format=yuyv422&framerate=20&video_size=800x600&video=/dev/video0#raw=timestamp#video=h264

ffmpeg:
  timestamp: "-vf \"drawtext=fontfile=DroidSansMono.ttf: text='%{localtime}': x=w-tw-10: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099\"" 

Here's the setup that doesn't (note the #hardware thing):

streams:
  linux_usbcam: ffmpeg:device?input_format=yuyv422&framerate=20&video_size=800x600&video=/dev/video0#raw=timestamp#video=h264#hardware

ffmpeg:
  timestamp: "-vf \"drawtext=fontfile=DroidSansMono.ttf: text='%{localtime}': x=w-tw-10: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099\"" 

Changing the order to #video=h264#hardware#raw=timestamp doesn't seem to fix anything.

I suspect this is due to the way go2rtc calls ffmpeg:

DBG [exec] run rtsp args=["ffmpeg","-hide_banner","-v","error","-hwaccel","vaapi","-hwaccel_output_format","vaapi","-hwaccel_flags","allow_profile_mismatch","-f","v4l2","-input_format","yuyv422","-framerate","20","-video_size","800x600","-i","/dev/video0","-vf","drawtext=fontfile=DroidSansMono.ttf: text='%{localtime}': x=w-tw-10: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099","-c:v","h264_vaapi","-g","50","-bf","0","-profile:v","high","-level:v","4.1","-sei:v","0","-an","-vf","format=vaapi|nv12,hwupload,scale_vaapi=out_color_matrix=bt709:out_range=tv:format=nv12","-user_agent","ffmpeg/go2rtc","-rtsp_transport","tcp","-f","rtsp","rtsp://127.0.0.1:8554/b49cff100b545443b381c966c660abb9"]

Making this ffmpeg call more verbose reveals that only the last -vf gets used, so that my drawing code is ignored:

Multiple -filter/-af/-vf options specified for stream 0, only the last option '-filter:v format=vaapi|nv12,hwupload,scale_vaapi=out_color_matrix=bt709:out_range=tv:format=nv12' will be used.

Is that a bug or am I doing something wrong? Please advise.

[AlexxIT]

Why you not using built-in timestamp preset?
https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-ffmpeg

[SkyWriter]

Oh, I didn't quite grasp it, sorry! Without the leading # it looked more like a suggestion to pass raw arguments directly to ffmpeg.

I tried it now, but unfortunately it doesn't seem to work this way either. Here are the two specs I have tried:

• ffmpeg:device?input_format=yuyv422&framerate=20&video_size=800x600&video=/dev/video0#drawtext=x=2:y=2:fontsize=12:fontcolor=white:box=1:boxcolor=black#video=h264#hardware
• ffmpeg:device?input_format=yuyv422&framerate=20&video_size=800x600&video=/dev/video0#video=h264#hardware#drawtext=x=2:y=2:fontsize=12:fontcolor=white:box=1:boxcolor=black

Both result in the same error:

15:39:08.932 DBG [exec] run rtsp args=["ffmpeg","-hide_banner","-v","error","-hwaccel","vaapi","-hwaccel_output_format","nv12","-hwaccel_flags","allow_profile_mismatch","-f","v4l2","-input_format","yuyv422","-framerate","20","-video_size","800x600","-i","/dev/video0","-c:v","h264_vaapi","-g","50","-bf","0","-profile:v","high","-level:v","4.1","-sei:v","0","-an","-vf","drawtext=x=2:y=2:fontsize=12:fontcolor=white:box=1:boxcolor=black:text='%{localtime\\:%Y-%m-%d %X}',hwupload","-user_agent","ffmpeg/go2rtc","-rtsp_transport","tcp","-f","rtsp","rtsp://127.0.0.1:8554/c565f17d4a8485513bc8075aefd36b2d"]
15:39:11.042 DBG [exec] [h264_vaapi @ 0x33d8f000] No usable encoding profile found.
15:39:11.042 DBG [exec] [vost#0:0/h264_vaapi @ 0x33d9cb80] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x33d9f600] Error sending frames to consumers: Function not implemented
15:39:11.043 DBG [exec] [vf#0:0 @ 0x33d9f600] Task finished with error code: -38 (Function not implemented)
[vf#0:0 @ 0x33d9f600] Terminating thread with return code -38 (Function not implemented)

I suspect this has to do with the same -vf option:

• What works (without the timestamp): "-vf","format=vaapi|nv12,hwupload,scale_vaapi=out_color_matrix=bt709:out_range=tv:format=nv12"
• What doesn't (with the built-in option): "-vf","drawtext=x=2:y=2:fontsize=12:fontcolor=white:box=1:boxcolor=black:text='%{localtime\\:%Y-%m-%d %X}',hwupload"

Can that be so?

[AlexxIT]

You may have a complicated situation here. When a raw image comes in. Then timestamp is drawn on it and then everything is encoded to h264 by hardware. Probably this kind of pipeline is not supported yet.

[SkyWriter]

Gotcha! Thank you for taking your time to look into it!

I ended up going full manual exec:ffmpeg... way, and it works a treat. Looks not as pretty, but at the end of the day one only has to do it once, hehe.

I suppose we can either close the issue, or turn it into a future feature request.

I appreciate the hard work!

[AlexxIT]

Please show your working version. I will add support when have time.

[SkyWriter]

Of course. Not sure if it's particularly useful since I'm using h264_qsv, but nonetheless here's what makes it tick for me:

streams:
  living_room: "exec:ffmpeg -f v4l2 -input_format yuyv422 -framerate 5 -video_size 800x600 -i /dev/video0 -vf \"drawtext=fontfile=DroidSansMono.ttf: text='%{localtime}': x=w-tw-10: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099\" -c:v h264_qsv -f rtsp {output}"