Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding Non-Steam Games Does Not Use The Right userdata Path with Multiple Accounts #1140

Closed
WIFIDarthMaul opened this issue Jul 10, 2024 · 17 comments · Fixed by #1141
Closed

Adding Non-Steam Games Does Not Use The Right userdata Path with Multiple Accounts #1140

WIFIDarthMaul opened this issue Jul 10, 2024 · 17 comments · Fixed by #1141
Labels
bug Something isn't working Command line Issues related to using SteamTinkerLaunch command line options Non-Steam Game Issues relating to Non-Steam Games launched through the Steam Client Steam Deck Issues related to using SteamTinkerLaunch on Steam Deck

Comments

@WIFIDarthMaul
Copy link

WIFIDarthMaul commented Jul 10, 2024 •
edited
Loading

System Information

  • SteamTinkerLaunch version: Latest
  • Distribution: Arch Linux

Issue Description

Like the title states, I am having trouble adding Non-Steam Games to Steam via commandline (Konsole). I have been using Steamtinkerlaunch for quite some time now and was using it to add games via .sh and it worked great for the longest time but now no longer does. Any help?
@WIFIDarthMaul WIFIDarthMaul added the bug Something isn't working label Jul 10, 2024
@sonic2kk
Copy link
Owner

sonic2kk commented Jul 10, 2024 •
edited
Loading

Are you using the Steam Client Beta out of interest? It did make some changes but it shouldn't have affected anything with shortcuts.

Also, did this happen after updating SteamTinkerLaunch and if so, do you know what the last working commit was? It worked for me about a week ago, after the initial Steam Client Beta update.

You will also need to provide a log of an attempted execution where you add a Non-Steam Game (you removed this section of the issue template). This may give some insight into what is going wrong.

@sonic2kk sonic2kk added the Non-Steam Game Issues relating to Non-Steam Games launched through the Steam Client label Jul 10, 2024
@sonic2kk
Copy link
Owner

sonic2kk commented Jul 10, 2024 •
edited
Loading

Just tested, addnonsteamgame still works over here. A log will be needed to investigate the problem. Any information you can add about when it stopped working might be helpful, too, to understand if this was may have been caused by a Steam update, a SteamTinkerLaunch update, or something else.

I also just tested by removing my shortcuts.vdf file entirely to see if perhaps the format had changed but was not impacting existing shortcuts.vdf files for some reason. This is not the case, a fresh shortcuts.vdf works perfectly fine. I will try one treated by Steam and will try writing into it, by removing the existing file, adding a Non-Steam Game via Steam, and then attempting to write into that file. I expect this to work, as on startup, the Steam Client does sanity checks on this file and some other files too.

@WIFIDarthMaul
Copy link
Author

WIFIDarthMaul commented Jul 10, 2024 via email

@sonic2kk
Copy link
Owner

sonic2kk commented Jul 10, 2024 •
edited
Loading

12.0 is very, very old. It's best to always use the latest commit to master, especially on SteamOS.

I'm on Steam Deck (Arch) btw

The Steam Deck is very different from a development target perspective than vanilla Arch Linux, please do not conflate the two and be explicit when opening issues about the operating system you are using.

SteamOS is no longer tested against and I will be removing advertisement of official support come v14.0.

Please test this on the Linux Desktop.

I tried the latest release and it doesn't work for me too.

If you are not using the latest commit to the master branch, you should try that.

It must me some Steam update.

The Steam Client Beta (on Linux Desktop) and Steam Deck Beta (specific to SteamOS), while usually close, are in fact different in various ways.

if I am understanding correctly, this stopped working even back on v12.0. It is probably a good idea to check master, as major changes have been made to Steam Shortcuts in that time.

@sonic2kk
Copy link
Owner

sonic2kk commented Jul 10, 2024 •
edited
Loading

I just tested with a fresh shortcuts.vdf with no shortcuts, created by Steam. I then added a game from Steam, and used SteamTinkerLaunch to add a game. When restarting the Steam Client, both games (the one added from Steam and the one added by SteamTinkerLaunch) were still present in my library.

This may be an issue specific to v12.0, which was released in December of 2022, or with v12.12, which was released in March of 2023, both being extremely outdated and given there have been numerous changes to Non-Steam Games since then, it's possible that something was fixed in master that Steam now breaks on with those older versions. #902, maybe #942, and #951 come to mind, but in particular, #965 could be something here. I was surprised Steam didn't bork the file because these were missing to begin with. It is possible that now it is doing that.

I cannot reproduce this issue so far, if you have any other specific cases you'd like me to try, let me know. A log file from an attempted add of a Non-Steam Game when running from master will help too.

@sonic2kk sonic2kk added the Steam Deck Issues related to using SteamTinkerLaunch on Steam Deck label Jul 10, 2024
@WIFIDarthMaul
Copy link
Author

WIFIDarthMaul commented Jul 11, 2024 •
edited
Loading

I tried the master and still no luck. Here's the log generated:
steamtinkerlaunch.log

@sonic2kk
Copy link
Owner

I don't think that's a log from right after running the Add Non-Steam Game command, it looks like it's from the Steam Client startup where it validates compatibility tools:

Wed Jul 10 07:13:24 PM CDT 2024 INFO - main - Checking command line: incoming arguments 'run /home/deck/.local/share/Steam/ubuntu12_32/../bin/d3ddriverquery64.exe -d3d12'
Wed Jul 10 07:13:24 PM CDT 2024 INFO - main - Checking command line: first argument 'run'

@WIFIDarthMaul
Copy link
Author

I don't think that's a log from right after running the Add Non-Steam Game command, it looks like it's from the Steam Client startup where it validates compatibility tools:

Wed Jul 10 07:13:24 PM CDT 2024 INFO - main - Checking command line: incoming arguments 'run /home/deck/.local/share/Steam/ubuntu12_32/../bin/d3ddriverquery64.exe -d3d12'
Wed Jul 10 07:13:24 PM CDT 2024 INFO - main - Checking command line: first argument 'run'

Whoops. I updated the log link

@sonic2kk
Copy link
Owner

sonic2kk commented Jul 11, 2024 •
edited
Loading

Thanks. I'm not seeing anything immediately jumping out in the log. The start command looks correct and it looks as though the function succeeds.

This is a shot in the dark: Are you using multiple Steam accounts, or did you add a new Steam account to your Steam Deck recently? If so, it's possible SteamTinkerLaunch is writing to the wrong shortcuts file.

  • You can get your Steam UserID by opening the file at ~/.local/share/Steam/config/loginusers.vdf. Here there will be a list of users logged into the Steam Client on this account.
  • Look for the block with your AccountName. The number above the opening curly brace for that block should be quite long. This is your Long UserID. You can convert it to a short ID by running the following command from Konsole, replacing 11111111 with your Account ID. echo $(( 11111111 & 0xFFFFFFFF )). I recommend copying and pasting it to make sure the result is accurate. Running commands from a random person on the Internet is scary and I don't believe in doing that, so I'll try to explain what this does. Feel free to verify on your own as well. You don't need to provide any of the actual IDs here to me, this is purely for if you want to verify this for yourself.
    • echo basically just tells Bash, your shell, to print something out into the console window.
    • The $(( )) part is how Bash, the shell, denotes arithmetic operations. It tells Bash "interpret everything here as arithmetic and then give it back to me." So whatever is computed inside of $(( )) is returned as a number. For example, echo $(( 1 + 1 )) would print out 2.
      -The 11111111 & 0xFFFFFFFF bit is bitwise arithmetic to convert the number given into a different format. Bitwise arithmetic is something I don't fully understand, but it has been reverse engineered by people much smarter than me to know this is how you convert down to the, iirc, 32bit unsigned integer that Steam uses. We use this arithmetic for other things too like calculating the Non-Steam Game AppID.
  • Once you have your short User ID, let's illustrate that it's 123456, you can that User ID and go to this folder, ~/.local/share/Steam/userdata/123456/config, replacing 123456 with your shortened User ID that you just got back. This folder contains the shortcuts.vdf file and some other files (such as Non-Steam Game grids and localconfig.vdf that stores some other general per-user Steam Client configuration).
  • Check the date that the shortcuts.vdf was modified. If it was not very recently (i.e. when you last ran the command), then SteamTinkerLaunch is writing to the wrong location. From the log, I can see that Steam is writing to /home/deck/.local/share/Steam/userdata/1520722864/config/shortcuts.vdf. If this is not the right path then something is probably up here.
    • You could do some more tests, such as closing the Steam Client and re-running the SteamTinkerLaunch command, and then checking if SteamTinkerLaunch actually wrote into the file (did the number of bytes change, did the modified date change, etc - This can be seen from the "Details" view in the Dolphin file manager, the third button to the right after the forward/backward buttons, or Ctrl+3 on a physical keyboard).

If you do not have multiple accounts... Then I am not sure :-)

As a courtesy note (there could be several valid intentional reasons for this, just want to mention) that you are using --usesteamgriddb, new in master, but are not passing a SteamGridDB API key. You can find out how to set this up here. I was curious if this could potentially cause an issue (such as trying to write an invalid icon into shortcuts.vdf) but it worked even without a SteamGridDB API key and with an invalid one.

@WIFIDarthMaul
Copy link
Author

WIFIDarthMaul commented Jul 11, 2024 via email

@sonic2kk
Copy link
Owner

This is something that has been in the back of my mind for SteamTinkerLaunch for a while (I've probably mentioned it buried in other issues by now) but we do need a way to specify the User ID. The default when no ID is passed should be to use the last logged in user. Right now SteamTinkerLaunch doesn't look at loginusers.vdf at all, it just guesses the directory by I think taking the one with the lowest value that is not 0 or ac but I am not sure. Instead we should take the User ID of the user in the block with "MostRecent" "1". This was not done before probably because parsing VDF files with Bash is a pain but has been implemented since then to read and write to config.vdf and localconfig.vdf specifically for Steam Shortcut-related functionality.

For specifying custom IDs - such as if you're logged into another account but have the Steam option enabled to prompt you for which account youu want to use, you might want to write into a specific user's shortcuts.vdf without having to open Steam and log in first, then restart Steam - adding it on the commandline is easy depending on the route taken; I'm unsure whether it should be direct ID or username. I am very much leaning towards ID but getting the short ID is not easy, so then it would have to be the long ID, but it's not even straightforward to get that without looking at loginusers.vdf yourself. The trouble with going with username is that they aren't guaranteed to be unique between accounts on the same computer. It is a bit tricky and I have other Non-Steam Game things I am looking to finalise, but it is something I have already planned to take a look at.

And once it's added to the commandline it can be exposed as an option on the UI fairly easily (using the username and User ID in a combobox entry in case we can't read loginusers.vdf properly for whatever reason).

If this issue is related to the last logged in user ID being guessed wrong, then unfortunately there is not a workaround currently, but this weekend I could try looking into changing the default behaviour to use the most recently logged in user which is one step in the right direction.

@sonic2kk sonic2kk added Command line Issues related to using SteamTinkerLaunch command line options and removed Non-Steam Game Issues relating to Non-Steam Games launched through the Steam Client labels Jul 11, 2024
@WIFIDarthMaul
Copy link
Author

WIFIDarthMaul commented Jul 11, 2024 via email

@sonic2kk
Copy link
Owner

sonic2kk commented Jul 11, 2024 •
edited
Loading

Heh, I call it intuition.

The full fix for this is two parts:

  1. Fix how we get the Steam userdata folder, because right now it is not actually basing it on anything other than directory order. I think a reasonable default is the last logged in account. This won't cover all cases, but I think it is a reasonable default, at least better than what we have now. To my understanding there is no way to mark an account as a "main" account so the most recent one is probably the way to go. Let me know if you have any other suggestions!
    • I am a bit concerned that this could add significant startup time to SteamTinkerLaunch as parsing this file each time could be expensive. That is, if we still store it at startup. We might've only did that for Depressurizer, which is deprecated (Remove Depressurizer Support #1029). We may only store the userdata path when working with shortcuts, which would negate performance concerns as this is not done every time SteamTinkerLaunch is started.
  2. Allow specifying the Steam User / User ID / both (possibly tricky but we could infer it with regex) in the Add Non-Steam Command, and so also on the UI as well. Usage could be something along these lines, where not specifying at all will result in the Most Recent account being used: steamtinkerlaunch ansg --user 123456.
    • One potential option is to just have the one --user flag, and if it is given and is not the current default User ID, we can parse the Long User ID (the one at the top of each block), the Short User ID (the one you got from the command), the "AccountName" (the name used to log in), and the "PersonaName" (the display name), probably in a loop. Then we can check these in order to see if they were passed into the command (ex: --user GabeN would first check if it's the Long ID, the Short ID, the Account Name, or the Persona Name. Doing this in a loop means we can exit early, and checking if we got the default user's Short ID first could give a speedup as reading these files can be a bit heavy if they are large (although they would probably be as large as something like config.vdf, tens of thousands of lines).

No ETA on implementation, for now as a desparate workaround you could symlink your main account's shortcuts.vdf to the alt account's shortcuts location. That way STL will be writing into the symlink. Then if your alt ever needs to use shortcuts you can just rename the files, basically "juggling" them. It's not ideal but it'll get you up and going until I can take a look at this. Hopefully I can sort the first part soon.

I will rename this issue title and try to get a look at implementing the first part, using the most recently logged in Steam User as the default, this weekend.

@WIFIDarthMaul
Copy link
Author

WIFIDarthMaul commented Jul 11, 2024 via email

@sonic2kk sonic2kk changed the title Adding Non-Steam Games Via Commandline No Longer Works Adding Non-Steam Games Does Not Use The Right User Account With Multiple Accounts Jul 11, 2024
@sonic2kk sonic2kk changed the title Adding Non-Steam Games Does Not Use The Right User Account With Multiple Accounts Adding Non-Steam Games Does Not Use The Right userdata Path with Multiple Accounts Jul 11, 2024
@sonic2kk sonic2kk added the Non-Steam Game Issues relating to Non-Steam Games launched through the Steam Client label Jul 11, 2024
@sonic2kk
Copy link
Owner

Very interestingly it seems you can add a STEAMUSERID=123456 to the global config file (at ~/.config/steamtinkerlaunch/global.conf, you can put it at the bottom of the file) and use the Short UserID as the value, and SteamTinkerLaunch will use this as the user directory. I don't believe this is exposed on the UI but that is another possible workaround if the symlinking ever fails.

Relevant code: https://github.com/sonic2kk/steamtinkerlaunch/blob/b48b32f91d15de5ef35b1749ef6f35d999bb7916/steamtinkerlaunch#L647-L649

I took a quick look today and it seems there is no way currently to loop through single sections of a VDF file so we'll probably have to do that. We can get values from specific sections, and get single sections, but nothing for generic sections from a browse of the codebase. But I'll try to work on this tomorrow.

@sonic2kk
Copy link
Owner

This took significant brainpower to figure out a way I was happy with resolving this, due to some limitations of what we can store and parse in Bash given a lack of data structures. I came up with an approach that I'm happy with on the use-mostrecent-loginuser-userdatadir.

That branch right now has code to allow us to use the MostRecent logged in Steam User's userdata folder instead of picking a random one.

  • Parse the contents of loginusers.vdf into a CSV stored in /dev/shm/steamtinkerlaunch called LoginUsersCSV.txt (to match convention)
    • This CSV only generated once, as the loginusers.vdf contents is unlikely to change while /dev/shm/steamtinkerlaunch is still cached. If it does change, the user will need to reboot or manually clear /dev/shm/steamtinkerlaunch which is pretty typical behaviour when things like this change, such as when a new Proton version is added.
    • The columns in this file are "Long UserID", "Short UserID", "MostRecent". In future if we need to we can store more data here, but for now, this is enough (Long UserID is not even fully necessary and is even unavailable in some cases)
    • Storing it in a CSV means we can access it in other parts of the program, such as in future when we add this to Add Non-Steam Game to allow it to select a Steam User
  • The CSV filling logic is basically:
    • Try to parse the data from loginusers.vdf
    • If we don't have that file or couldn't parse it, then as a fall back, we use the old method of just picking the first result of the find command against the Steam userdata directory, and then write out Long UserID=1, Short UserID={the found folder name}, MostRecent=1 (because we only have 1 user in the case of no loginusers.vdf, because we only pick one result from the find command -- We don't pick multiple because if even if we did we would have no way of knowing who the most recent Steam User is if we don't have loginusers.vdf)
    • If we didn't find anything in the userdata folder to fall back to, then assume no users are logged in and log a warning to expect problems because STL relies on these!
  • We fill out this CSV when we call setSteamPaths which means it is filled out whenever SteamTinkerLaunch is executed but only done once if the file does not exist or if it is blank
  • In setSteamPaths this is where before we had the logic to just pick the first result of the find command. Now we instead loop through the contents of the above CSV file
    • We take the Short UserID and the value of MostRecent from each line in the file
    • If the current line has a MostRecent Steam User, then use this Steam User's ShortID as the current Steam UserID, and set the path to the Steam User's userdata folder to the base Steam userdata root folder path plus this UserID (i.e. with a UserID of 12345, set the Steam User userdata path to ~/.local/share/Steam/userdata/12345).
  • Since the fallback to using the find command is done in the function to fill the Proton CSV, we don't have to do it here. If we don't have a Steam UserID or Steam User userdata path after this, log a warning as well.
  • Doing the fallback in the Fill Proton CSV function means that, under normal circumstances, we should always have at least one Steam User, which means we don't have to do any extra checks anywhere else we might expect the LoginUsersCSV to exist. If we need it we can just call setSteamPaths as a safety and afterwards we can assume that we'll have a reference to at least one Steam User.

I would've preferred a more generic way to parse through and get the blocks from a VDF file, but there is no generic way to do this. So instead we just get the entire "users" section of the loginusers.vdf file and then parse out known patterns matching \t"[0-9]+" which matches every line with one indent that has a series of 1 or more numbers in double quotes, i.e. like "432435490" (which is a known pattern that our loginusers.vdf block start with inside of the "users" block. We can do this because we know the structure of loginusers.vdf. Sadly I didn't figure out a good way to do this generically and to know the difference between a block and a value, so this is a semi-hardcoded solution for loginusers.vdf based on the known structure of the file.

I have some a good amount of testing on this over the last few hours, it works reasonably well for me, but I need to do a bit more testing and then I will open up a PR. I don't think it'll get merged tonight unless I'm feeling brave, so sorry about the delay on this as I did say I wanted to resolve this over the weekend. Since this is a reasonably significant change to a core piece of functionality of SteamTinkerLaunch I want to make extra sure that this does not break finding the userdata folder. I am fairly sure it won't as if the new logic fails we should just fall back to the old logic, but I want to be as close to sure of that as possible before merging.

Thanks for the patience on this 🙂

@sonic2kk sonic2kk mentioned this issue Jul 17, 2024
Merged
3 tasks
@sonic2kk
Copy link
Owner

This took me way too long to get around to testing fully and merging, sorry about that. But this should finally be merged in and ready with SteamTinkerLaunch v14.0.20240727-1 and above. Now SteamTinkerLaunch will use the last logged in user.

If you switch user accounts without rebooting, you will need to remove /dev/shm/steamtinkerlaunch.

There is still pending work to allow selecting the Steam User when adding Non-Steam Games, but now the default will be the last logged in user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working Command line Issues related to using SteamTinkerLaunch command line options Non-Steam Game Issues relating to Non-Steam Games launched through the Steam Client Steam Deck Issues related to using SteamTinkerLaunch on Steam Deck
Projects
None yet
Milestone
No milestone
2 participants
@sonic2kk @WIFIDarthMaul