Background Game Recording while in gamescope-session often crashes the client after quitting games

[matte-schwartz]

Your system information

• Steam client version (build number or date): 1731990050
• Distribution (e.g. Ubuntu): Arch Linux, Fedora, SteamOS
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes
• Steam Logs: steam-logs.tar.gz (from ROG Ally, Arch Linux)
• GPU: AMD Z1 Extreme, AMD Vangogh APU

Please describe your issue in as much detail as possible:

When Game Recording in the background is enabled within the Steam client, and gamescope-session is in use, quitting games will often lead to an apparent driver deadlock that restarts the Steam client. So far, I've had it happen twice on SteamOS Main on my Steam Deck OLED, and then it happens far more frequently on my ROG Ally currently running CachyOS (Arch Linux).

[2024-11-20 08:49:05] Encoding complete
[2024-11-20 08:49:05] *** pw_stream_destroy called from wrong context, check thread and locking: Operation not permitted
[2024-11-20 08:49:05] *** impl_ext_end_proxy called from wrong context, check thread and locking: Operation not permitted
[2024-11-20 08:49:05] 'this->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:62 do_unlock()
[2024-11-20 08:49:05] CDesktopCapturePipeWire: PipeWire stream state changed to paused
[2024-11-20 08:49:05] CDesktopCapturePipeWire: PipeWire stream state changed to unconnected
[2024-11-20 08:49:15] Driver deadlock in hardware accelerated desktop capture, aborting
[2024-11-20 08:49:15] src/steamUI/gamestream/desktopstreamthread.cpp (148) : Driver deadlock in hardware accelerated desktop capture, aborting
[2024-11-20 08:49:15] src/steamUI/gamestream/desktopstreamthread.cpp (148) : Driver deadlock in hardware accelerated desktop capture, aborting
[2024-11-20 08:49:15] src/steamUI/gamestream/desktopstreamthread.cpp (148) : Fatal assert; application exiting
[2024-11-20 08:49:15] src/steamUI/gamestream/desktopstreamthread.cpp (148) : Fatal assert; application exiting
[2024-11-20 08:49:15] 11/20 08:49:15 Init: Installing breakpad exception handler for appid(steam)/version(1731990050)/tid(1567)
[2024-11-20 08:49:15] assert_20241120084915_86.dmp[4904]: Uploading dump (out-of-process)
[2024-11-20 08:49:15] /tmp/dumps/assert_20241120084915_86.dmp
[2024-11-20 08:49:16] assert_20241120084915_86.dmp[4904]: Finished uploading minidump (out-of-process): success = yes
[2024-11-20 08:49:16] assert_20241120084915_86.dmp[4904]: response: CrashID=bp-495d9995-2eb3-4503-a8ea-ee9122241120
[2024-11-20 08:49:16] assert_20241120084915_86.dmp[4904]: file ''/tmp/dumps/assert_20241120084915_86.dmp'', upload yes: ''CrashID=bp-495d9995-2eb3-4503-a8ea-ee9122241120''

The ROG Ally runs a much newer Mesa than SteamOS, but other than that I tested the same exact gamescope and Steam client versions across both devices. I've also seen reports from elsewhere about this issue on non-SteamOS distros: ublue-os/bazzite#1862. When I did experience the same Steam crash on SteamOS Main, console output looked identical to the logging above.

Steps for reproducing this issue:

1. Start running a game (I chose Hades II)
2. Let the game play for ~30 seconds or so
3. Exit the game by using the left-side Steam menu

[matte-schwartz]

Seems to happen to me 100% of the time on SteamOS staging, maybe some other package version that hasn't made it to Stable/Preview/Main yet that's present on staging and upstream Arch.

Can rule out Mesa since downgrading to SteamOS' Mesa version makes no difference.

[matte-schwartz]

I seem to be able to avoid a full Steam client crash even with the driver deadlock warning if I add -debug_steamapi to my Steam launch options. I also added LIBVA_MESSAGING_LEVEL=2 and LIBVA_TRACE=/tmp/libva
to /etc/libva.conf

logs with debugging verbosity + `-debug_steamapi`

GameAction [AppID 1145350, ActionID 1] : LaunchApp changed task to Completed with ""
[gamescope] [Error] pipewire: push_pipewire_buffer: Already had a buffer?!: Resource temporarily unavailable
[gamescope] [Error] pipewire: push_pipewire_buffer: Already had a buffer?!: Resource temporarily unavailable
>>> Switching video stream from NONE to Desktop_MovieStream
[gamescope] [Error] pipewire: push_pipewire_buffer: Already had a buffer?!: Resource temporarily unavailable
Adding process 17586 for gameID 1145350
Refreshing process list for game 1145350
2 game pids rooted at pid 17581:
- 17581,17586
GameScope focus changed to Steam
CDesktopCapturePipeWire: updating capture appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: setting gamescope focus appid: 1145350
CDesktopCapturePipeWire: PipeWire stream state changed to paused
[gamescope] [Info]  pipewire: stream state changed: paused
Building new pipeline
Trying to create an encoder for recording: [hardware_enabled=true][hardware_suppressed=false]
Allowed Codecs: 4
Failed to load optional module libnvidia-encode.so.1
Failed to load optional module libamfrt32.so.1
Created encoder VAAPI for codec 4
CDesktopCapturePipeWire: PipeWire stream format changed (size: 7680x2160, format 23, stride 7680, size 24883200, dmabuf: 0)
CDesktopCapturePipeWire: PipeWire stream format changed (size: 3840x1080, format 23, stride 3840, size 6220800, dmabuf: 0)
CGameStreamVideoStageVAAPI: Reinitializing 3840x1080 at 12000 Kbit/s and 60.00 FPS with quality 1
CDesktopCapturePipeWire: PipeWire stream state changed to streaming
[gamescope] [Info]  pipewire: stream state changed: streaming
[gamescope] [Info]  xwm: Exposing appid 1145350 (1145350 32-bit) focus-wise on pipewire stream.
Video Encoder Changed
>>> Saw first frame, starting a session
Adding process 17587 for gameID 1145350
Refreshing process list for game 1145350
3 game pids rooted at pid 17581:
- 17581,17586,17587
Game Recording - Recording Started [recording=bg_1145350_20241121_185304]
GameScope focus changed to Steam
>>> Recording new resolution 3840x1080 using bitrate 12000kbps
>>> Capture method set to Desktop PipeWire NV12 + VAAPI H264
>>> First video pts is 724.604747s, first video PTS is 0.033333s
...
src/steamUI/gamestream/desktopstreamthread.cpp (148) : Driver deadlock in hardware accelerated desktop capture, aborting
src/steamUI/gamestream/desktopstreamthread.cpp (148) : Driver deadlock in hardware accelerated desktop capture, aborting
src/steamUI/gamestream/desktopstreamthread.cpp (148) : Driver deadlock in hardware accelerated desktop capture, aborting
11/21 10:53:39 Init: Installing breakpad exception handler for appid(steam)/version(1732151988)/tid(15981)
assert_20241121105339_46.dmp[19554]: Uploading dump (out-of-process)
/tmp/dumps/assert_20241121105339_46.dmp
>>> Stopped desktop stream
PulseAudio: Context connection terminated
Game Recording - Recording Stopped [recording=bg_1145350_20241121_185304]
Stopped game recording session
------------------------------------------------------------------------------------------------------
Game Recording - game stopped [gameid=1145350]

[lostgoat]

Thanks for the report. There’s a fix queued up on the client side. It won’t be directly on the next beta, but probably the one after that.