Installation part of the workflow is slow (by 2 orders of magnitude)

[joshka]

TL;DR:
Expected <30s for release process
Actual 5min
Consider taiki-e/install-action instead of action-install-gh-release.

---

In many of my crate repos I have CI that install various cargo tools using https://github.com/taiki-e/install-action. This is fast (usually single digit seconds for multiple tools).

The install process in release-plz-action is 2 orders of magnitude slower (~4-5 minutes). I suspect perhaps the difference is down to whatever is happening with the caching? If my crate isn't getting updated as often as release-plz / cargo-semver, then I never benefit from the cache and always feel the downside of how long it takes to upload the cache.

taiki-e install-action has a list of supported tools (cargo-semver isn't there yet, and neither is release-plz), but it also falls back to using cargo binstall if the tools aren't available, which is also fairly fast. It doesn't need to cache the install as the metadata about release versions is written to the git repo and doesn't have to be parsed for each install.

Example logs (turn on timestamps):
release-plz using action-install-gh-release: https://github.com/joshka/tui-big-text/actions/runs/9265763819/job/25488487114
ci process using taiki install-action: https://github.com/joshka/tui-big-text/actions/runs/9265763820/job/25488487509

Proposed fix:

• Use taiki-e/install-action
• Add release-plz to taiki-e/install-action
• Add cargo-semver to taiki-e/install-action
  (The process for adding a tool is to write a small amount of metadata, run a tool that checks the available versions once and then let the build system handle updates automatically from then on)

[marcoieni]

Hi Josh! 👋
Thanks for opening this issue. I also experienced the same delay the first time I install release-plz on a repo or when a new release-plz version is out. Let's fix this 🎉

I suspect perhaps the difference is down to whatever is happening with the caching?

I don't know. Maybe action-install-gh-release is doing a lot of calls to the GitHub API and that's why it takes so long. 🤷‍♂️
Either we spend time debugging it, or we go in another direction.

I'm fine using cargo-binstall, since Release-plz already has cargo-binstall instructions, and running cargo-binstall is reproducible locally.

Install-action vs cargo-binstall

What's the advantage of using install-action over the cargo-binstall GitHub action? 🤔
Also, if we add release-plz to install-action, does it mean I need to wait for the next release of install-action to have the latest release-plz available in GitHub Actions? 🤔 If yes, this doesn't work for me, as I want to ship immediately to users.

It doesn't need to cache the install as the metadata about release versions is written to the git repo and doesn't have to be parsed for each install.

What do you mean? I don't understand what is saved and in which git repo 🤔

Using cargo-binstall

If we use cargo-binstall, for caching we can rely on release-plz/release-plz#1442 as it should cache release-plz and cargo-semver-checks. However, this has the disadvantage of removing the files from the cache when Cargo.lock changes. So maybe it's more efficient to cache the binaries ourselves, maybe with https://github.com/actions/cache
The cargo-binstall action doesn't check if the binary is already present.
For now, it's fine if cargo-binstall is not cached (Do you think installing cargo-binstall with install-action has better performance because of some caching?). Or maybe we can download cargo-binstall (i.e. invoke the github action) only if not present in cache.

We could run cargo-binstall to install release-plz in the following way:

cargo binstall release-plz \
  --version ${{ inputs.version }}
  --strategies crate-meta-data

In the same way, we could install cargo-semver-checks like this:

cargo binstall cargo-semver-checks \
  --version 0.31.0
  --strategies crate-meta-data

We should also pass the GITHUB_TOKEN environment variable to avoid rate limiting.

Question: is cargo-binstall faster if we pass the --git flag? Maybe it doesn't fetch crates.io 🤔

Very curious to hear your thoughts on this! 🤗

[joshka]

What's the advantage of using install-action over the cargo-binstall GitHub action? 🤔

1.6K (binstall) vs 6.9K (install-action) installs in repos. install-action uses binstall, but has a path that allows caching that github metadata once in the install-action repo rather than hitting the api for every run.

Also, if we add release-plz to install-action, does it mean I need to wait for the next release of install-action to have the latest release-plz available in GitHub Actions? 🤔 If yes, this doesn't work for me, as I want to ship immediately to users.

The manifests are updated every three hours. https://github.com/taiki-e/install-action/blob/main/.github/workflows/ci.yml
The action falls back to using cargo-binstall if a specified version is not in the manifest(e.g. you release release-plz 1.2.3, but only 1.2.2 is in the manifest, anyone wanting a non-specific version will see the update less than 3 hours. Anyone requesting [email protected] will see the release immediately.)

I don't know. Maybe action-install-gh-release is doing a lot of calls to the GitHub API and that's why it takes so long. 🤷‍♂️
...
What do you mean? I don't understand what is saved and in which git repo 🤔

The metadata that I was referring to that is cached by install-action is the github api calls mentioned above (the version info and download links). Instead of every action run querying the release status API for every installation, it runs with the values that are cached from the install-action repo (example: https://github.com/taiki-e/install-action/blob/main/manifests/cargo-audit.json)

For now, it's fine if cargo-binstall is not cached (Do you think installing cargo-binstall with install-action has better performance because of some caching?). Or maybe we can download cargo-binstall (i.e. invoke the github action) only if not present in cache.

I wouldn't worry about caching any of this and investing the effort to duplicate the work of install-action, which falls back on using cargo-binstall anyway. Often this action takes literally no time at all: Example - this is from tui-scrollview: installing cargo-llvm-cov (https://github.com/joshka/tui-scrollview/actions/runs/9169759107/job/25210765142)

Pulling the cargo deps for this repo takes about 2 secs and updating takes 7 secs. install-action takes 0 secs and 0 effort.

[joshka]

Testing this out in #146, the install time goes down to 2s without any specific changes to the install-action repo.

[marcoieni]

Ok, I think I understand how the install-action works now.

If we add support for release-plz and cargo-semver-checks, install-action periodically will update their metadata, so that you can avoid using cargo-binstall, and save calling crates.io and GitHub API.
I.e. if you request version 1.2.3, it knows the GitHub Release url and checksum.

Probably we need to add support for release-plz and cargo-semver-checks in install-action.
Otherwise the performance is the same as running cargo-binstall action, right?

Anyway, I merged the PR since it brings significant improvements, thanks 🙌

I just want to make sure I understand everything happening under the hood.

Would you like to add support for release-plz and cargo-semver-checks in install-action? 👀
Otherwise I can also do it 👍

[marcoieni]

Released this. Thanks for addressing this issue ❤️

[joshka]

Ok, I think I understand how the install-action works now.

Yep - your summary is correct.

Would you like to add support for release-plz and cargo-semver-checks in install-action? 👀 Otherwise I can also do it 👍

Sure thing :)