Command to remove outdated packages/binaries

[reitzig]

Feature Overview

As a (power 😇) user, I want to maintain my user-space, cross-project tooling with aqua without blowing up storage usage over time.

I understand that an automatism is hard to conceive of, due to the highly dynamic and semi-stateless way aqua works. However, a command for me to call would solve my use case; I would just integrate it into my upgrade routine.

Why is the feature needed?

When I frequently upgrade tools installed through aqua, it assembles quite a volume of files. After just a few days of usage, I have:

🐧❯ du -sh ~/.local/share/aquaproj-aqua/
1,2G	/home/raphael/.local/share/aquaproj-aqua/

Meanwhile, a clean install results in

🐧❯ aqua list -a --installed | wc -l
21
🐧❯ du -sh ~/.local/share/aquaproj-aqua
862M	/home/raphael/.local/share/aquaproj-aqua

With a few rather active tools in my list, I expect to need a cleanup routine sooner rather than later.

Workaround

While I can definitely

aqua rm --all

and rely on lazy install, depending on when/where the install triggers, the experience is less than pretty. For instance, with direnv/direnv and starship/starship:

🐧❯ aqua rm --all
INFO[0000] removing all packages                         aqua_version=2.29.0 env=linux/amd64 program=aqua
/home/raphael/.local/share/aquaproj-aqua/pkgs/github_release/github.com/direnv/direnv/v2.34.0/direnv.linux-amd64/direnv.linux-amd64: command not found
- (line 3): 
        "/home/raphael/.local/share/aquaproj-aqua/pkgs/github_release/github.com/direnv/direnv/v2.34.0/direnv.linux-amd64/direnv.linux-amd64" export fish | source;
        ^
in function '__direnv_export_eval'
in event handler: handler for generic event “fish_prompt”
INFO[0000] download and unarchive the package            aqua_version=2.29.0 env=linux/amd64 exe_name=starship package_name=starship/starship package_version=v1.19.0 program=aqua registry=standard
[WARN] - (starship::modules::custom): Executing custom command "env | grep -c KUBE" timed out.
[WARN] - (starship::modules::custom): You can set command_timeout in your config to a higher value or set ignore_timeout to true for this module to allow longer-running commands to keep executing.

I guess I could do this as an "atomic" action:

aqua rm --all && aqua install -a

But then I would always re-download all tools, which seems a waste of time and bandwidth.

Example Code

I imagine we could have something like:

$ aqua rm --outdated

This would delete all versions except the latest ones.

(This would, of course, cause re-installs for all projects that pin an older version. For me, that'd be acceptable.)

Note sure how one would encode "delete all versions but the ones pinned in the global config" -- --all means different things to aqua install and aqua rm. 🤔

Maybe a dedicated command like

$ aqua clean [--all]

could delete all versions but the once references in the current scope?

Note

I found this, which seems to leave room for future discussions:

You may want to keep the latest version and delete only old versions, but we don't support it now.
Lazy Install will reinstall the latest version.
#538

So here I am. 😁

[suzuki-shunsuke]

Thank you for your proposal.

I think this proposal makes sense.

I think we need to do the following things.

1. Search the directory AQUA_ROOT_DIR and get the list of installed packages and their versions
2. Compare versions and get the latest version by each package
3. Delete old versions

https://aquaproj.github.io/docs/reference/registry-config/#package-types

• 🟢 cargo
• 🟢 github_archive
• 🟢 github_content
• 🟢 github_release
• 🟢 go_build
• ❌ go_install
• ❌ http

Another solution is to use a machine with enough storage lol.

[suzuki-shunsuke]

Related proposal:

• https://github.com/orgs/aquaproj/discussions/3086

[NikitaCOEUR]

A related implementation is proposed in #3442

[reitzig]

aqua vacuum looks nice, thank you!

One question: After upgrading and aqua vacuum --init, is this what I should see?

~
❯ tree .local/share/aquaproj-aqua/pkgs/github_release/github.com/starship/starship/
.local/share/aquaproj-aqua/pkgs/github_release/github.com/starship/starship/
├── v1.19.0
│   └── starship-x86_64-unknown-linux-musl.tar.gz
│       └── starship
├── v1.20.1
│   └── starship-x86_64-unknown-linux-musl.tar.gz
│       └── starship
├── v1.21.1
│   └── starship-x86_64-unknown-linux-musl.tar.gz
│       └── starship
└── v1.22.1
    └── starship-x86_64-unknown-linux-musl.tar.gz
        └── starship

9 directories, 4 files

~
❯ tree .local/share/aquaproj-aqua/metadata/pkgs/github_release/github.com/starship/starship/
.local/share/aquaproj-aqua/metadata/pkgs/github_release/github.com/starship/starship/
└── v1.22.1
    └── starship-x86_64-unknown-linux-musl.tar.gz
        └── timestamp.txt

3 directories, 1 file

Meaning, there are four old versions around, but only the newest got a timestamp.

The docs say:

aqua vacuum --init can't record date times of install packages which are not found in aqua.yaml. If you want to record their date times, you need to remove them by aqua rm command and re-install them.

So even after running vaccuum --init on all aqua.yaml on my system, all old versions that are not referenced anywhere anymore won't be cleaned up, ever? 🤔

If so, Operation Burn Everything seems inevitable.

[suzuki-shunsuke]

Yes, this is the current limitation.
But you only need to run aqua rm --all only once.

Or if you can list old versions, you can generate aqua.yaml based on the list and run vacuum --init.

Now aqua doesn't have the feature to list all installed package versions.

[reitzig]

Understood, thanks

I was trying to avoid aqua rm --all for the reasons stated in the OP; seems I'll have to, this one last time. :D

Or if you can list old versions

If I could do that, I could xarg aqua rm them. ;) Unfortunately, I don't think I can (easily), can I?

(I've also yet to find a way to list all commands (not packages) provided by any given aqua config -- but that's a whole different issue I'll have to think about a bit more. ;))

[suzuki-shunsuke]

Unfortunately, I don't think I can (easily), can I?

Yes, I think so too.
If we can achieve this easily, aqua vacuum --init would support all package versions. 😅