Make fetching source tarball by git commit more robust

[Flamefire]

(created using eb --new-pr)

I had a crate that was supposed to fetch a git commit from https://github.com/astral-sh/reqwest-middleware. However the specified git commit 21ceec9a5fd2e8d6f71c3ea2999078fecbd13cbe was not available for git checkout after the clone.

My understanding is, that this commit is not reachable from the default (main) branch that is fetched, possibly after same rebase.

The approach done here is to add a separate git fetch call to fetch the specified commit. That resolves the issue in my case.

We could go even further and don't use git clone at all:

mkdir repo
cd repo
git init
git fetch --depth=1 https://github.com/... sha/branch/tag [--recursive]
git checkout sha/branch/tag

[Micket]

I also think this checkout only works until github garbage collects. This dangling commit it just held on by reflog, and will disappear with a git gc. So, 90 days or so and it's gone i think.

I am in favor of just fetching what we need for the requested checkout, but it wouldn't actually solve the underlying issue; UV devs are throwing away their old patches. I think the devs here should tag/branch their commit and not leave it detached, and the old method would still work (I think). So the cost here would be that we wouldn't detect this issue until the source is truly forever gone.

Note: I think you meant reqwest-middleware as that's the onethat has the commit you mention here:
astral-sh/reqwest-middleware@21ceec9

(edit: I could be wrong, maybe github never GC's? But i doubt that)

[Flamefire]

I also think this checkout only works until github garbage collects. This dangling commit it just held on by reflog, and will disappear with a git gc. So, 90 days or so and it's gone i think.

Not sure: I locally created a commit, amended and ran git gc --aggressive --force --prune=now but was still able to access the original commit. So maybe it does work even after any GC/Prune?

Note: I think you meant reqwest-middleware as that's the onethat has the commit you mention here: astral-sh/reqwest-middleware@21ceec9

Yes, mis-pasted. Fixed in the description

[Micket]

I think it does go out, unless github sets up their reflog expiry date to something infinite.
https://git-scm.com/docs/git-gc#Documentation/git-gc.txt-gcreflogExpire

Testing it out:

mkdir test-gc
cd test-gc
git init
echo "First version" > file.txt
git add file.txt
git commit -m "First version"
echo  "Modified version" > file.txt
git add file.txt
git commit --amend -m "Modified version"
git reflog   # both show up and i can checkout either

git reflog expire --expire=now --all  # default is 90 days but i don't want to wait
git gc
git switch ca00a7a  # now gives me "fatal: invalid reference: ca00a7a"

Just gc isn't enough, the reglof need to expire before that's cleaned up.

[Flamefire]

Ah ok, so the reflog needs to be GCed.

I guess in that case we cannot do anything. At least the change makes the checkouts faster and gives us 90 days more time.

[Micket]

Yes the UV devs should at least just add a git tag "reqwests-uv-2.x" on their patch commit, that would prevent it from expiring.

[Flamefire]

It isn't actually uv but a dependency of it. And that repo doesn't has an issue tracker so I could complain. Also the commit is currently only in a branch, not somewhere in main, i.e. not merged yet. So can't help that for now.

[Micket]

But it's the UV devs patching their dependencies is it not? (on several repos under https://github.com/astral-sh)

[Flamefire]

Ah it's the same GitHub org, haven't seen that. I opened an issue in the uv repo