Custom Command: Allow Using Steam Linux Runtime with Custom Commands

[sonic2kk]

Adds the ability to use the Steam Linux Runtime with Custom Commands. This should help with compatibility, as Proton is not intended to be used to run games without the Steam Linux Runtime.

This functionality should improve compatibility with custom commands, especially Proton custom commands. This should also mean that, effectively, using a custom command, with ONLY_CUSTOMCMD enabled, and with the Steam Linux Runtime option from this PR enabled, allows the custom command to act as a full replacement for the default Steam executable. There are some caveats and limitations (no integration with Steamworks as usual, mod managers won't respect this, etc) but in my testing it acts as a smooth drop-in replacement strategy now even in edge-cases where this didn't work before.

This currently works, but there is no option for this on the UI, it is hardcoded with __DEBUG_CUSTCMD_SLR in the code.

---

Note: The implementation here may serve as a basis for using the SLR with Winetricks, see #860.

---

TODO:

• Add checkbox for toggling Steam Linux Runtime
  • Needs tested!!!!
• Add support for Native Linux SLR (currently only Scout 1.0, as 3.0 has slightly different logic needed to fetch, would need handled sepatately as this PR only handles integrating the current SLR logic)
• Update langfiles
• Version bump

---

EDIT: The checkbox has been added, and the option for this (CUSTOMCMD_USESLR) will be enabled by default. This is because it is supposed to improve compatibility. Proton also requires the SLR; many things work without it on some systems but it's still meant to be used. With Native Linux commands, it is unnecessary, but can still be very useful except for things like scripts.

So for broad compatibility, this is enabled by default, but can be disabled in cases where it causes issues. For example it may get in the way of inter-process communication, and it would prevent scripts from accessing things on PATH (as it would be isolated to the container).

The tooltip notes the compatibility benefit and briefly illustrates some limitations.

[sonic2kk]

Tested and this works with native games now without breaking Proton games! The log is also correct, it will log when it has a native game and when it's building a log for a Proton game.

I confirmed the Steam Linux Runtime is used by checking for additional pressure-vessel-* processes that appear when launching custom commands with this PR, and verified that without this PR those processes disappear.

Now we have to implement this option on the UI.

[sonic2kk]

Replaced the hardcoded debug var __DEBUG_CUSTCMD_SLR with a config variable CUSTOMCMD_USESLR.

This PR is untested with this change.

[sonic2kk]

Tested with SA-Mod-Manager as a Custom Command for Sonic Adventure 2, the SLR is enabled/disabled as expected.

[sonic2kk]

Wine custom commands cannot be ran for native Linux titles as there is no compat data folder for them to run inside of, forgot about that...

So testing Proton custom commands with Native titles doesn't work, but that is to be expected.

[sonic2kk]

Verified that this worked with To the Moon where a custom command was set as the game start script and a different native Linux game in a different directory. The Steam Linux Runtime was enabled and disabled as expected.

[sonic2kk]

I think testing for this PR is dev complete now, the functionality seems to work. It is in very good shape from my testing.

The main things left are:

• Retouch the logic for extProtonRun SLR fetching, can it be as simple as the logic for Native Linux SLR in launchCustomProg?
  • I wrote the logic in extProtonRun quite a while back, but wrote the logic in launchCustomProg with a fresh mind, so perhaps I need to revisit extProtonRun's implementation (I left a comment about this in the code as well)
• Give the implementation here another pass to make sure it's fine
• Update langfiles & version bump as usual

[sonic2kk]

In future we should look into integrating the Crash Guesser, or some kind of Crash Guesser, with custom commands IF ONLY_CUSTOMCMD is enabled. A custom program that I'm testing with, SAModManager, sometimes fails to start with Wine. It would be nice to quickly be able to retry this.

We should also look into integrating PROTON_LOG for custom commands. This is not super important but it would give a little bit more Proton insight, and would be some extra sugar on top of the goal of making ONLY_CUSTOMCMD essentially replace the game start command but with some more flexibility.

[sonic2kk]

Did some more testing to make sure this played nicely with regular custom command usage (Custom Command, then game after Custom Command exits), Fork Custom Command, and Inject Custom Command. I realised most of my testing last night was for Only Custom Command.

I tested this with a retouch of the extProtonRun logic for setting the Steam Linux Runtime and it works fine, it is much cleaner now! I will push this shortly.

The last thing I want to test a bit more is native custom programs with Proton games; the inverse doesn't work - you can't run Proton custom commands with native games because native games don't have a prefix; we could theoretically generate one but that's probably unnecessary. I don't see a case either where you'd need to run a Windows binary to do something with a native title.

But the case of running a native custom command with a Proton game does exist. I think it works fine, and logically it should work because launchCustomProg will check if an executable is Windows or not, and if it's not Windows it will fall through in launchCustomProg. But doesn't hurt to test some more.

[sonic2kk]

Running custom programs with Proton will not use the native SLR but the Proton one. Seems we will have to force this in some way.

setSLRReap is fetching the SLR based on the same logic that it would for Proton titles, because all of the set variables are for Proton. We need to find a way to make setSLRReap (or rather, setNonGameSLRReap which calls it for us) to get the native SLR. Probably the simplest way is to give it a flag.

[sonic2kk]

We could actually just refactor for force Proton var in setSLRReap and setNonGameSLRReap to take a value of 2 to state that it should force native. We could make it SLRFORCETYPE.

Going to test such a refactor locally and see if it works.

[sonic2kk]

Seems like I got it working locally.

• Native standalone games still use the right SLR
• Proton standalone games still use the right SLR
• Native games with native custom commands still use the right SLR, forked, injected, and used together
• Proton games with Proton custom commands still use the right SLR, forked, injected, and used together
• Proton games with native custom commands are now correctly forced to use the native SLR, and the Proton game still runs the SLR fetcher and correctly gets the required SLR when forked/launched after a native custom command

Will push the latest changes in a bit.

[sonic2kk]

Functionality has been tested pretty extensively and it seems to work without introducing regressions, implementation is all good as well from another scan. ShellCheck is green, langfiles are updated, version is bumped, this is good to merge!