Unable to update PSResourceGet via Artifactory remote repo

[sean-r-williams]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.

Steps to reproduce

• Configure Artifactory with a remote repository targeting PSGallery.
• Connect PSResourceGet to this repository. (Because PSGallery is v2-only, the connection to Artifactory within PSResourceGet must also be v2).
• Update PSResourceGet with Update-PSResource Microsoft.PowerShell.PSResourceGet.

Expected behavior

PS> Update-PSResource Microsoft.PowerShell.PSResourceGet
PS> # update completes without issues

Actual behavior

PS> update-psresource Microsoft.PowerShell.PSResourceGet -Repository Artifactory
update-psresource : Package does not exist on the server
At line:1 char:1
+ update-psresource Microsoft.PowerShell.PSResourceGet -Repository Arti ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Microsoft.Power...lets.FindHelper:FindHelper) [Update-PSResource], ResourceNotFoundException
    + FullyQualifiedErrorId : FindNameConvertToPSResourceFailure,Microsoft.PowerShell.PSResourceGet.Cmdlets.UpdatePSResource

Error details

No response

Environment data

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Binary     1.0.4.1               Microsoft.PowerShell.PSResourceGet  {Find-PSResource, Get-InstalledPSResource, Get-PSResourceRepository, Get-PSScriptFileInfo…}


Name                           Value
----                           -----
PSVersion                      7.4.2
PSEdition                      Core
GitCommitId                    7.4.2
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

This seems to be the combination of two separate issues:

• PSResourceGet changes behavior in calls like FindPackageById() if we detect the upstream repo is a JFrog repo. After discussing with JFrog support, they confirmed that NuGet calls like this are being passed unfettered to the upstream. This means that PSGallery is receiving subtly different API call parameters when it's being reflected through Artifactory.
• PSGallery, if provided with the exact filter string IsLatestVersion, is returning unexpected results. I filed filter IsLatestVersion returns invalid response for package Microsoft.PowerShell.PSResourceGet PowerShellGallery#273 to track this.

[sean-r-williams]

I see three options here:

• Adjust the behavior of PSGallery s.t. $filter=IsLatestVersion works as expected for this package (i.e. fix filter IsLatestVersion returns invalid response for package Microsoft.PowerShell.PSResourceGet PowerShellGallery#273)
• Inside of PSResourceGet, revisit how/where JFrog quirk flags are being used (we may no longer need them for all scenarios anymore)
• Munge the filter used (i.e. add a space or similar) to work around whatever behavior is occurring within PSGallery

[sean-r-williams]

@alerickson @SydneyhSmith This bug isn't something I can directly contribute a fix for since we're running up against [unexpected] behavior in the Gallery.

Can someone from your team take a look here?

We originally engaged JFrog support for this and have some line of dialogue with their R&D. (JFrog folks, see ticket #296792) If we need to get adjustments made in JFrog to align its behavior with other servers (thus obviating the need for quirk flags in this space), now would be an excellent time to do so.

Alternatively, if your team is comfortable with munging the filter (i.e. convert from IsLatestVersion to IsLatestVersion eq true, add a space to the front of the string, etc.), I can cut a PR for that (with the understanding that it will decrease legibility of the codebase).

[anamnavi]

As mentioned in this issue, we're going to look into the Gallery codebase to see what is happening and if we can't safely make the change there then we'll open a fix client side.

[sean-r-williams]

@SydneyhSmith In PowerShell/PowerShellGallery#273 you mentioned a client-side fix was likely going to be the lowest-friction solve. It's been over a month and I haven't seen any movement - is your team still working on this? If there's bandwidth constraints preventing prioritization/delivery of said fix, can someone from your team describe what the intended client-side fix was?

I'm happy to provide a PR implementing option #3 described in #1656 (comment), but this feels a bit kludgy- I want to avoid adding more cruft to the codebase than is necessary.

[sean-r-williams]

@SydneyhSmith @anamnavi It's been over 3 months and there hasn't been any movement on this issue nor PowerShell/PowerShellGallery#273.

As I described in PowerShell/PowerShellGallery#273 (comment), this issue is now occurring on other modules in the Gallery (e.g. JiraPS). @alerickson's previous assessment (see PowerShell/PowerShellGallery#273 (comment)) suggested this was only when a lower-version stable release was uploaded after a higher-version prerelease - JiraPS did not do this sequence of uploads. It only released a higher-version prerelease after a stable-version release, which is a far more frequent/standard approach. The scope of impact appears to be broader than originally described.

I suspect that most PSGallery customers are not using Artifactory, so exposure of this issue is getting attenuated.

Right now, PSResourceGet+Artifactory is still not production-ready. Having packages randomly disappear from visibility means we can't use PSRG for provisioning infrastructure, nor can we reasonably expect internal PowerShell teams (who may not have deep expertise in NuGet API semantics) to use PSRG+Artifactory for their deployment needs.

If your team does not have resources to prioritize a fix, it would be appreciated if you could provide some clarity on the intended client-side fix discussed back in June (PowerShell/PowerShellGallery#273 (comment)) so that the community can implement it.

[anamnavi]

@sean-r-williams my apologies for the delay in getting back to this. We had looked into the PSGallery side fix but decided it'll be simpler to fix client-side with PSResourceGet. I've opened up this PR: #1761

We tested the JiraPS API call you shared in the comment here and tweaked it with the filter change proposed in my PR and it returns the package correctly. However we don't have an Artifactory feed of our own to thoroughly test against, so could you kindly test my PRs change against your Artifactory feed and see if the fix resolves the issue?

[sean-r-williams]

@anamnavi Thanks for your patience - was on vacation last week.

I noticed comments in the PR that trying to repro this with ADO yields an error with filter IsLatestVersion eq true (vs IsLatestVersion). What is ADO's behavior in the broader context of this bug? If you create an Azure Artifacts feed with PSGallery as an upstream NuGet feed, connect PSRG to that feed, then Find-PSResource Microsoft.PowerShell.PSResourceGet, are you able to repro same issue as Artifactory through ADO?

A "yes" here would mean we're in a a rock+hard-place situation with ADO, but we're already looking at cross-provider interaction behavior at the moment with this issue+PR. It would be good to understand whether ADO will have a similar class of problems as Artifactory did (alongside other package management providers).