Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(referrers): push manifest with subject #163

Merged
merged 29 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ internal static void CheckOciSubjectHeader(this HttpResponseMessage response, Re
if (repository.ReferrersState == Referrers.ReferrersState.Unknown && response.Headers.Contains("OCI-Subject"))
{
// Set it to Supported when the response header contains OCI-Subject
repository.ReferrersState = Referrers.ReferrersState.Supported;
repository.SetReferrersState(true);
}

// If the "OCI-Subject" header is NOT set, it means that either the manifest
Expand Down
2 changes: 1 addition & 1 deletion src/OrasProject.Oras/Registry/Remote/ManifestStore.cs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

public class ManifestStore(Repository repository) : IManifestStore
{
public Repository Repository { get; init; } = repository;

Check warning on line 32 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / Analyze (8.0.x)

Parameter 'Repository repository' is captured into the state of the enclosing type and its value is also used to initialize a field, property, or event.

Check warning on line 32 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / Analyze (8.0.x)

Parameter 'Repository repository' is captured into the state of the enclosing type and its value is also used to initialize a field, property, or event.

Check warning on line 32 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / build (8.0.x)

Parameter 'Repository repository' is captured into the state of the enclosing type and its value is also used to initialize a field, property, or event.

/// <summary>
/// Fetches the content identified by the descriptor.
Expand Down Expand Up @@ -248,10 +248,10 @@
desc.Annotations = imageManifest.Annotations;
break;
default:
return;

Check warning on line 251 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View check run for this annotation

Codecov / codecov/patch

src/OrasProject.Oras/Registry/Remote/ManifestStore.cs#L251

Added line #L251 was not covered by tests
}

Repository.ReferrersState = Referrers.ReferrersState.NotSupported;
Repository.SetReferrersState(false);
await UpdateReferrersIndex(subject, new Referrers.ReferrerChange(desc, Referrers.ReferrerOperation.Add), cancellationToken).ConfigureAwait(false);
}

Expand Down Expand Up @@ -304,7 +304,7 @@
}

// 4. delete the dangling original referrers index, if applicable
await DeleteAsync(oldDesc, cancellationToken).ConfigureAwait(false);

Check warning on line 307 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / Analyze (8.0.x)

Possible null reference argument for parameter 'target' in 'Task ManifestStore.DeleteAsync(Descriptor target, CancellationToken cancellationToken = default(CancellationToken))'.

Check warning on line 307 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / Analyze (8.0.x)

Possible null reference argument for parameter 'target' in 'Task ManifestStore.DeleteAsync(Descriptor target, CancellationToken cancellationToken = default(CancellationToken))'.

Check warning on line 307 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View workflow job for this annotation

GitHub Actions / build (8.0.x)

Possible null reference argument for parameter 'target' in 'Task ManifestStore.DeleteAsync(Descriptor target, CancellationToken cancellationToken = default(CancellationToken))'.
}

/// <summary>
Expand All @@ -323,8 +323,8 @@
var (desc, content) = await FetchAsync(referrersTag, cancellationToken).ConfigureAwait(false);
var index = JsonSerializer.Deserialize<Index>(content);
Wwwsylvia marked this conversation as resolved.
Show resolved Hide resolved
if (index == null)
{
throw new JsonException("null index manifests list");

Check warning on line 327 in src/OrasProject.Oras/Registry/Remote/ManifestStore.cs

View check run for this annotation

Codecov / codecov/patch

src/OrasProject.Oras/Registry/Remote/ManifestStore.cs#L326-L327

Added lines #L326 - L327 were not covered by tests
pat-pan marked this conversation as resolved.
Show resolved Hide resolved
}
return (desc, index.Manifests);
}
Expand Down
18 changes: 11 additions & 7 deletions src/OrasProject.Oras/Registry/Remote/Referrers.cs
Original file line number Diff line number Diff line change
Expand Up @@ -70,46 +70,50 @@
updateRequired = true;
continue;
}

var basicDesc = oldReferrer.BasicDescriptor;
if (updatedReferrersSet.Contains(basicDesc))
if (referrerChange.ReferrerOperation == ReferrerOperation.Delete && Equals(basicDesc, referrerChange.Referrer.BasicDescriptor))
{
// Skip any duplicate referrers
updateRequired = true;
continue;
}
// Update the updatedReferrers list
// Add referrer index in the updatedReferrersSet
if (referrerChange.ReferrerOperation == ReferrerOperation.Delete && Descriptor.Equals(basicDesc, referrerChange.Referrer.BasicDescriptor))

if (updatedReferrersSet.Contains(basicDesc))
{
// Skip any duplicate referrers
updateRequired = true;
continue;
}

// Update the updatedReferrers list
// Add referrer into the updatedReferrersSet
updatedReferrers.Add(oldReferrer);
updatedReferrersSet.Add(basicDesc);
}

var basicReferrerDesc = referrerChange.Referrer.BasicDescriptor;
if (referrerChange.ReferrerOperation == ReferrerOperation.Add)
{
var basicReferrerDesc = referrerChange.Referrer.BasicDescriptor;
if (!updatedReferrersSet.Contains(basicReferrerDesc))
{
// Add the new referrer only when it has not already existed in the updatedReferrersSet
pat-pan marked this conversation as resolved.
Show resolved Hide resolved
updatedReferrers.Add(referrerChange.Referrer);
updatedReferrersSet.Add(basicReferrerDesc);
updateRequired = true;
}
}

// Skip unnecessary update
if (!updateRequired && updatedReferrersSet.Count == oldReferrers.Count)
if (!updateRequired)
{
pat-pan marked this conversation as resolved.
Show resolved Hide resolved
// Check for any new referrers in the updatedReferrersSet that are not present in the oldReferrers list
foreach (var oldReferrer in oldReferrers)
{
var basicDesc = oldReferrer.BasicDescriptor;
if (!updatedReferrersSet.Contains(basicDesc))
{
updateRequired = true;
break;

Check warning on line 116 in src/OrasProject.Oras/Registry/Remote/Referrers.cs

View check run for this annotation

Codecov / codecov/patch

src/OrasProject.Oras/Registry/Remote/Referrers.cs#L114-L116

Added lines #L114 - L116 were not covered by tests
}
}

Expand All @@ -117,7 +121,7 @@
{
return (updatedReferrers, false);
}
}

Check warning on line 124 in src/OrasProject.Oras/Registry/Remote/Referrers.cs

View check run for this annotation

Codecov / codecov/patch

src/OrasProject.Oras/Registry/Remote/Referrers.cs#L124

Added line #L124 was not covered by tests
return (updatedReferrers, true);
}
}
5 changes: 5 additions & 0 deletions src/OrasProject.Oras/Registry/Remote/Repository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -368,4 +368,9 @@ internal Reference ParseReferenceFromContentReference(string reference)
/// <returns></returns>
public async Task MountAsync(Descriptor descriptor, string fromRepository, Func<CancellationToken, Task<Stream>>? getContent = null, CancellationToken cancellationToken = default)
=> await ((IMounter)Blobs).MountAsync(descriptor, fromRepository, getContent, cancellationToken).ConfigureAwait(false);

public void SetReferrersState(bool isSupported)
pat-pan marked this conversation as resolved.
Show resolved Hide resolved
{
ReferrersState = isSupported ? Referrers.ReferrersState.Supported : Referrers.ReferrersState.NotSupported;
}
}
18 changes: 10 additions & 8 deletions src/OrasProject.Oras/Registry/Remote/RepositoryOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,15 @@ public struct RepositoryOptions
/// </summary>
public int TagListPageSize { get; set; }

// SkipReferrersGc specifies whether to delete the dangling referrers
// index when referrers tag schema is utilized.
// - If false, the old referrers index will be deleted after the new one is successfully uploaded.
// - If true, the old referrers index is kept.
// By default, it is disabled (set to false). See also:
// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#referrers-tag-schema
// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#pushing-manifests-with-subject
// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#deleting-manifests
/// <summary>
/// SkipReferrersGc specifies whether to delete the dangling referrers
/// index when referrers tag schema is utilized.
/// - If false, the old referrers index will be deleted after the new one is successfully uploaded.
/// - If true, the old referrers index is kept.
/// By default, it is disabled (set to false). See also:
/// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#referrers-tag-schema
/// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#pushing-manifests-with-subject
/// - https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md#deleting-manifests
/// </summary>
public bool SkipReferrersGc { get; set; }
pat-pan marked this conversation as resolved.
Show resolved Hide resolved
}
Loading