Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: Patch Installer Manifests #2497

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

2uasimojo
Copy link
Member

@2uasimojo 2uasimojo commented Oct 18, 2024

Add an enhancement doc proposing API changes to allow hive consumers to specify arbitrary edits to the manifests generated by openshift-install create manifests prior to feeding them to openshift-install create cluster.

HIVE-1793

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 18, 2024
Copy link
Contributor

openshift-ci bot commented Oct 18, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link
Contributor

openshift-ci bot commented Oct 18, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 2uasimojo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 18, 2024
Comment on lines +93 to +101
- `ManifestSelector` supports one subfield, `Glob`, which accepts a path matching string as
supported by golang's [filepath.Glob](https://pkg.go.dev/path/filepath#Glob).
- We will execute the glob relative to the working directory in which manifests were generated.
- For security, we will error on any glob starting with `/` or containing `../`.
- `ManifestSelector` is extensible.
E.g. in the future we may wish to match manifests based on the GVK of the object therein.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@patrickdillon I would really like your input here! What should the initial mechanism be for identifying the manifest(s) to be patched?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline with Patrick. We're going to go with file glob for the first pass.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@2uasimojo and I discussed over slack, but recording here for posterity. Eric identified filename (glob) or GVK as two options.

The installer has existing code which will deserialize all manifests (using filename globs) and convert them into k8s client.Objects using gvk:

https://github.com/openshift/installer/blob/9422034132089f395da68485f54a75ae52c32df7/pkg/asset/machines/clusterapi.go#L535-L574

That said, the advantages of using GVK are not immmediately obvious.

The manifest filenames should be rather stable, and we're brainstorming whether there are methods to make that even more guaranteed.

@2uasimojo
Copy link
Member Author

/cc @abraverm

Thought you might be interested to see how ClusterDeploymentCustomization is evolving :)

@openshift-ci openshift-ci bot requested a review from abraverm October 18, 2024 22:19
@abraverm
Copy link
Contributor

abraverm commented Oct 21, 2024

@2uasimojo 🥹

What is needed is a mechanism for the hive user to specify arbitrary modifications to OpenShift
manifests generated by installer.

With this enhancment and cluster template operator, a fully automated UPI might be possible.

- `ManifestSelector` is extensible.
E.g. in the future we may wish to match manifests based on the GVK of the object therein.

### `ClusterDeployment.Spec.Provisioning.CustomizationRef`
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-

We should fix up the clusterpool controller to automatically add this ref from the inventory to the CD. That way the clusterpool user could do inventory things in either/both ways (install-config and/or manifest patching). Note: requires it not to be a failure if a CDC reffed here contains no manifest patches? Or maybe we can be strict for non-pool clusters? Is it a problem if a non-pool CD uses installConfigPatches??

-Self

Copy link
Contributor

@abraverm abraverm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

**TBD.**

Where/how do we report the success/failure of
- CDC as a whole? In a new `CD.Status.Conditions[?Type=CustomizationsSuccessful]` or similar?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now the conditions are in CDC and CP, the later has a report for the status of all CDC in its inventory. CD has reference to the CDC itself, I don't know if CD should keep a copy of the error in its own conditions.

Where/how do we report the success/failure of
- CDC as a whole? In a new `CD.Status.Conditions[?Type=CustomizationsSuccessful]` or similar?
- Individial `CDC.InstallerManifestPatches`?
How are we doing it for `InstallConfigPatches` for ClusterPools?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CDC has ApplySucceededCondition, and any issue reported in the condition message.


- Is it a failure if `ManifestSelector` matches zero manifests?
- Is it a failure if a `Patches[].Path` matches no paths?
- Failure to execute a patch should probably bubble up as an overall failure.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of InstallConfigPatches, the CP resolution fails and the failure is saved in the CDC which causes CP to pick a different CDC in the next attempt.

Add an enhancement doc proposing API changes to allow hive consumers to
specify arbitrary edits to the manifests generated by `openshift-install
create manifests` prior to feeding them to `openshift-install create
cluster`.

HIVE-1793
@2uasimojo 2uasimojo marked this pull request as ready for review November 14, 2024 21:58
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 14, 2024
@2uasimojo
Copy link
Member Author

Finished this out alongside the code (#2499). Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants