RFC: Script updater

[Nick-Mazuk]

Script updater

The problem

Right now, the only way for users to update these scripts are by manually checking the website and redownloading them. It would be nice to let users update scripts directly in Finale.

The solution

I'd like to create a new script that allows users to check all the scripts downloaded and will tell them which scripts are out of date. They can then either update all of them or only certain scripts of their choosing.

The design

Design constaints

The challenge here is that version numbers or last-updated date are not sufficient. We cannot rely on script metadata because if a library function is updated, the bundled script is also updated. Since we don't keep track of which scripts are updated if a library function is updated, the metadata is not reliable.

Additionally, sometimes we accidentally break a script's functionality (because we don't have testing, see #255 and #260). And as these scripts are used by professionals on strict time constraints, they cannot afford to wait until we fix any broken scripts. As such, they need to be able to revert to the previous version of the script they downloaded.

Detailed design

Determining outdated scripts

To determine if a script is updated, we should check the file contents instead of the version numbers. If the contents of the downloaded version and the one on GitHub are the same, the script is not out of date. If they are different, the script is out of date. This would have to be done locally on a user's computer since that's the only place that knows the current versions of the user's installed scripts.

However, comparing the file contents itself is inefficient since the user would have to download potentially 100+ files from GitHub every time they check for updates. Instead, I propose we use hashes of the file contents and add that to the existing metadata file. That way, they just need to download a single file and can do the rest of the processing locally.

https://github.com/finale-lua/jw-lua-scripts-docs/blob/main/src/lib/lib/script-data.json

Additionally, we can simply compare the file names of the local scripts with those in this JSON file to determine which scripts are even hosted in the repository.

Restoring outdated scripts

Lua scripts can already read/write to a user's file system (see the configuration library)

For this, we'd need to know a scripts location on the file system. Then, we can create a subfolder called "restore_scripts" that'll hold all the previously downloaded versions of the scripts. We would then add a timestamp of when we updated the script to the file name:

my_script.lua => my_script_1659760130249.lua

That means when we want to restore a previous version, we can simply find most recently replaced version of each script and restore it. Multiple previous versions of each script can be kept here, allowing a user to go back in time as much as they need to get their system working again.

You can think of this system like a stack created from the file names:

my_script_1659760130249.lua
my_script_1659760000000.lua
my_script_1659750000000.lua

To restore a script, we just pop off the top item from the stack.

my_script_1659760000000.lua
my_script_1659750000000.lua

The downside with this approach is that while users can go back in time, they cannot go forward in time. They must go straight to the most recent version in the repo if they wish to go forward in time.

Considerations

1. File names will now be treated as unique IDs for each script. File names cannot be updated without breaking the updater functionality.
2. This would require updates to RGP Lua, mostly along the lines of what's described in Adding FCLuaScriptItem and FCLuaScriptItems to RGP Lua and JetStream Controller #287, with a few extra features (that discussion is in that other issue).
3. This would definitely open pandora's box of being able to read/write to other scripts (though arguably the configuration library already opened the box)

Alternatives

We could just keep doing what we're currently doing.

We could also move this feature directly into RGP Lua, though I'm hesitant to suggest adding features to RGP Lua that only help this repository.

cc @rpatters1 @ThistleSifter @cv-on-hub

[rpatters1]

The new security features have put an automatic script updater within inches of RGP Lua 0.68 (the forthcoming release.) All we really need do is add another finaleplugin property (e.g., ScriptUrl) and RGP Lua can prompt the user with any that need to be updated. It is not a solution specific to this repo, but only this repo would currently be using it. For scripts to be updated, they would have to have both ScriptUrl and HashUrl finaleplugin properties.