Have PackageVariant set readiness gate on PackageRevisions #156
Conversation
|
/test presubmit-nephio-go-test |
| @@ -334,6 +350,13 @@ func (r *PackageVariantReconciler) ensurePackageVariant(ctx context.Context, | |||
| if err = r.Client.Create(ctx, newPR); err != nil { | |||
| return nil, err | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Looks good. I'd be interested to see how this works in a full test-infra e2e run.
Also, may be somewhat aligned with what @kispaljr was working on?
|
/test presubmit-nephio-go-test |
5 similar comments
|
/test presubmit-nephio-go-test |
|
/test presubmit-nephio-go-test |
|
/test presubmit-nephio-go-test |
|
/test presubmit-nephio-go-test |
|
/test presubmit-nephio-go-test |
controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go
Outdated
Show resolved
Hide resolved
controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go
Outdated
Show resolved
Hide resolved
controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
In the findAndUpdateExistingRevisions() method, at line 473:
// Save the updated PackageRevisionResources
if err := r.updatePackageResources(ctx, prr, pv); err != nil {
return nil, err
}
The pipeline will also be executed here, so the Condition should be set according to the results.
There was a problem hiding this comment.
in updateDraft() at line 736:
err := r.Client.Update(ctx, draft)
if err != nil {
return nil, err
}
this is tricky, but technically this Update() call will also result in the re-execution of the pipeline, so we should record the results in the Condition. The tricky part is that I don't know how to get the rendering results, and without it, the best we can do is to report the error or success of Update() itself.
There was a problem hiding this comment.
Done both:
- the
findAndUpdateExistingRevisions()one could be a bit better by including the initial set of the Condition directly inprrinstead of doing a separater.Client.Update()- I want to look into automated (and a few more manual) tests first though - the
updateDraft()one took more time to work around to avoid ending up in an endless loop where doing theUpdate()to set the Condition would trigger the pipeline, resulting in a rendering result which we'd need to record in the Condition by doing anUpdate(), triggering the pipeline... you get the idea. :)- thanks to @liamfallon for suggesting we have the server detect and skip the pipeline when an update only sets the Condition/readiness info
- works as desired when creating a PackageVariant - unable to test in this update flow due to immediate Git conflicts as seen in #469
- PackageVariant controller now uses a readiness gate to allow
a PackageRevision to complete its mutation pipeline before it
is allowed to be proposed/approved
- refactored conversion of Kptfiles to YAML since readiness condition
information is stored in the package Kptfile
- unified all cases to the same kyaml/yaml-based method
(KptFile.ToYamlString() and ToYamlString(*fn.KubeObject))
- this ensures consistency in the YAML (indentation, field order etc.)
- and reduces the chances of Git conflicts when setting and updating
readiness conditions
- added more info to error message in case of Git conflict when applying
a patch
nephio-project/nephio#615
- previous solution broke existing functionality to copy comments between Kptfile objects
- resulting in failing unit tests
- fix for that brought back the Git conflicts caused by indentation, field order etc.
- re-refactored conversion of Kptfiles to YAML
- KubeObject-related conversion methods (KubeObject/YAML and YAML/KubeObject)
now live out in the util package as they aren't Kptfile-specific any more
nephio-project/nephio#615
JamesMcDermott
force-pushed
the
pv-pipeline-readiness-gates
branch
from
3cff6bb to
60cb107
Compare
… pipeline-passed condition
- review comments to increase applicability of pipeline-passed readiness condition
- to enable this: if an incoming PackageRevision update changes only that
readiness condition, detect that, skip the pipeline/rendering/etc., and
only update the Kptfile with the readiness condition update
- other tidy-up/wording comments
nephio-project/nephio#615
- also broke ground on testing PackageVariants in end-to-end tests at all nephio-project/nephio#615
- also broke ground on testing PackageVariants in end-to-end tests at all nephio-project/nephio#615
- tests now delete package variants whose upstream is a repo created by the test nephio-project/nephio#615
…on flows
- this includes both init and clone operations
- a new PackageRevision now has the PackagePipelinePassed Condition and
ReadinessGate by default
- set to False on creation, but the initial (no-op) render run effectively
sets it to True by default
- API-based init operations (Create) will include any Conditions passed in
in the request
- and add ReadinessGates based on the Conditions
- a successful render run sets the PackagePipelinePassed condition:
- to False before starting
- to True on successful completion
- effectively preventing the PackageRevision from being proposed using porchctl
rpkg propose while the pipeline is running
- a future commit will move this validation from porchctl into the API server
to lock out API-based propose operations as well
- PackageVariant controller sets its own condition and readiness gate on
PackageRevisions it manages
- and manages this Condition independently to allow it to lock out propose
operations while the controller is performing multiple operations
- PackageVariant controller can handle optimistic concurrency conflicts when
making its final (per-reconcile) status update to a PackageVariant
- some refactoring/renaming for readbility
- fixed typo in Makefile comment
nephio-project/nephio#615
…erver - apply readiness-gate lockout to API operations as well as porchctl CLI nephio-project/nephio#615
| @@ -516,6 +535,35 @@ func (cad *cadEngine) UpdatePackageResources(ctx context.Context, repositoryObj | |||
| return repoPkgRev, renderStatus, nil | |||
| } | |||
|
|
|||
| func pushPipelineReadinessGate(ctx context.Context, repo repository.Repository, repoPr repository.PackageRevision) error { | |||
There was a problem hiding this comment.
This makes the engine handling of drafts even more complex but there is no oopter way to do this with the current code structure. Let's refactor engine.go in Nephio R5.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JamesMcDermott, liamfallon The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
- new document describing operation of readiness gates on PackageRevisions
- in general (no existing document for it)
- specifically the changes introduced in
nephio-project/porch#156
- including default readiness gates managed by Porch server for all
PackageRevisions and by package variant controller for PackageRevisions
controlled by a PackageVariant
- new diagram to illustrate flows
nephio-project/nephio#615
nephio-project/porch#156
|
@JamesMcDermott: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Implements #615
incoming ReadinessGates and Conditions are now saved in PackageRevision creation flows
ReadinessGate by default
sets it to True by default
in the request
a successful render run sets the PackagePipelinePassed condition:
rpkg propose while the pipeline is running
to lock out API-based propose operations as well
PackageVariant controller now sets its own condition and readiness gate on PackageRevisions it manages
PackageVariant controller can now handle optimistic concurrency conflicts when making its final (per-reconcile) status update to a PackageVariant
refactors internal conversion of Kptfiles to YAML since readiness condition information is stored in the package Kptfile
adds more info to error message in case of Git conflict when applying a patch
CLI E2E tests now give more information on failure
CLI E2E tests now fail on first test failure to allow easier identification of the failed test's output
some refactoring/renaming for readbility
fixed typo in Makefile comment