Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Agent leader election config #349

Closed
wants to merge 1 commit into from
Closed

✨ Agent leader election config #349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions crdsv1beta1/0001_00_operator.open-cluster-management.io_klusterlets.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,9 @@ spec:
type: object
additionalProperties:
type: string
disableLeaderElection:
description: Disable leader election. Use the klusterlet agent default value if this field is not defined.
type: boolean
featureGates:
description: "FeatureGates represents the list of feature gates for registration If it is set empty, default feature gates will be used. If it is set, featuregate/Foo is an example of one item in FeatureGates: 1. If featuregate/Foo does not exist, registration-operator will discard it 2. If featuregate/Foo exists and is false by default. It is now possible to set featuregate/Foo=[false|true] 3. If featuregate/Foo exists and is true by default. If a cluster-admin upgrading from 1 to 2 wants to continue having featuregate/Foo=false, \the can set featuregate/Foo=false before upgrading. Let's say the cluster-admin wants featuregate/Foo=false."
type: array
Expand Down Expand Up @@ -185,6 +188,18 @@ spec:
type: integer
format: int32
default: 50
leaderElectionLeaseDuration:
description: The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
leaderElectionRenewDeadline:
description: The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
leaderElectionRetryPeriod:
description: The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
registrationDriver:
description: This provides driver details required to register with hub
type: object
Expand Down Expand Up @@ -267,6 +282,9 @@ spec:
description: AppliedManifestWorkEvictionGracePeriod is the eviction grace period the work agent will wait before evicting the AppliedManifestWorks, whose corresponding ManifestWorks are missing on the hub cluster, from the managed cluster. If not present, the default value of the work agent will be used.
type: string
pattern: ^([0-9]+(s|m|h))+$
disableLeaderElection:
description: Disable leader election. Use the klusterlet agent default value if this field is not defined.
type: boolean
featureGates:
description: "FeatureGates represents the list of feature gates for work If it is set empty, default feature gates will be used. If it is set, featuregate/Foo is an example of one item in FeatureGates: 1. If featuregate/Foo does not exist, registration-operator will discard it 2. If featuregate/Foo exists and is false by default. It is now possible to set featuregate/Foo=[false|true] 3. If featuregate/Foo exists and is true by default. If a cluster-admin upgrading from 1 to 2 wants to continue having featuregate/Foo=false, \the can set featuregate/Foo=false before upgrading. Let's say the cluster-admin wants featuregate/Foo=false."
type: array
Expand Down Expand Up @@ -295,6 +313,18 @@ spec:
type: integer
format: int32
default: 50
leaderElectionLeaseDuration:
description: The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
leaderElectionRenewDeadline:
description: The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
leaderElectionRetryPeriod:
description: The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. Use the klusterlet agent default value if this field is not defined.
type: string
pattern: ^([0-9]+(s|m|h))+$
workImagePullSpec:
description: WorkImagePullSpec represents the desired image configuration of work agent. quay.io/open-cluster-management.io/work:latest will be used if unspecified.
type: string
Expand Down
60 changes: 60 additions & 0 deletions operator/v1/0000_00_operator.open-cluster-management.io_klusterlets.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,11 @@ spec:
ClusterAnnotations is annotations with the reserve prefix "agent.open-cluster-management.io" set on
ManagedCluster when creating only, other actors can update it afterwards.
type: object
disableLeaderElection:
description: |-
Disable leader election.
Use the klusterlet agent default value if this field is not defined.
type: boolean
featureGates:
description: "FeatureGates represents the list of feature gates
for registration\nIf it is set empty, default feature gates
Expand Down Expand Up @@ -289,6 +294,31 @@ spec:
If it is set empty, use the default value: 50
format: int32
type: integer
leaderElectionLeaseDuration:
description: |-
The duration that non-leader candidates will wait after observing a leadership
renewal until attempting to acquire leadership of a led but unrenewed leader
slot. This is effectively the maximum duration that a leader can be stopped
before it is replaced by another candidate. This is only applicable if leader
election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
leaderElectionRenewDeadline:
description: |-
The interval between attempts by the acting master to renew a leadership slot
before it stops leading. This must be less than or equal to the lease duration.
This is only applicable if leader election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
leaderElectionRetryPeriod:
description: |-
The duration the clients should wait between attempting acquisition and renewal
of a leadership. This is only applicable if leader election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
registrationDriver:
description: This provides driver details required to register
with hub
Expand Down Expand Up @@ -407,6 +437,11 @@ spec:
the managed cluster. If not present, the default value of the work agent will be used.
pattern: ^([0-9]+(s|m|h))+$
type: string
disableLeaderElection:
description: |-
Disable leader election.
Use the klusterlet agent default value if this field is not defined.
type: boolean
featureGates:
description: "FeatureGates represents the list of feature gates
for work\nIf it is set empty, default feature gates will be
Expand Down Expand Up @@ -451,6 +486,31 @@ spec:
If it is set empty, use the default value: 50
format: int32
type: integer
leaderElectionLeaseDuration:
description: |-
The duration that non-leader candidates will wait after observing a leadership
renewal until attempting to acquire leadership of a led but unrenewed leader
slot. This is effectively the maximum duration that a leader can be stopped
before it is replaced by another candidate. This is only applicable if leader
election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
leaderElectionRenewDeadline:
description: |-
The interval between attempts by the acting master to renew a leadership slot
before it stops leading. This must be less than or equal to the lease duration.
This is only applicable if leader election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
leaderElectionRetryPeriod:
description: |-
The duration the clients should wait between attempting acquisition and renewal
of a leadership. This is only applicable if leader election is enabled.
Use the klusterlet agent default value if this field is not defined.
pattern: ^([0-9]+(s|m|h))+$
type: string
type: object
workImagePullSpec:
description: |-
Expand Down
44 changes: 44 additions & 0 deletions operator/v1/types_klusterlet.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,10 @@ type RegistrationConfiguration struct {
// +kubebuilder:default:=100
KubeAPIBurst int32 `json:"kubeAPIBurst,omitempty"`

// Leader election configuration
// +optional
LeaderElectionConfig `json:",inline"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it needs to be inline.


// BootstrapKubeConfigs defines the ordered list of bootstrap kubeconfigs. The order decides which bootstrap kubeconfig to use first when rebootstrap.
//
// When the agent loses the connection to the current hub over HubConnectionTimeoutSeconds, or the managedcluster CR
Expand Down Expand Up @@ -269,6 +273,10 @@ type WorkAgentConfiguration struct {
// +kubebuilder:default:=100
KubeAPIBurst int32 `json:"kubeAPIBurst,omitempty"`

// Leader election configuration
// +optional
LeaderElectionConfig `json:",inline"`

// AppliedManifestWorkEvictionGracePeriod is the eviction grace period the work agent will wait before
// evicting the AppliedManifestWorks, whose corresponding ManifestWorks are missing on the hub cluster, from
// the managed cluster. If not present, the default value of the work agent will be used.
Expand All @@ -278,6 +286,42 @@ type WorkAgentConfiguration struct {
AppliedManifestWorkEvictionGracePeriod *metav1.Duration `json:"appliedManifestWorkEvictionGracePeriod,omitempty"`
}

// Leader election configuration
type LeaderElectionConfig struct {
// Disable leader election.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need to expose every configurations here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the leader election will always be enabled in OCM single node case, and we are not 100% sure if it will impact the user's cases, it might be useful to provide a disable option in Klusterlet.

// Use the klusterlet agent default value if this field is not defined.
// +optional
DisableLeaderElection bool `json:"disableLeaderElection,omitempty"`

// The duration that non-leader candidates will wait after observing a leadership
// renewal until attempting to acquire leadership of a led but unrenewed leader
// slot. This is effectively the maximum duration that a leader can be stopped
// before it is replaced by another candidate. This is only applicable if leader
// election is enabled.
// Use the klusterlet agent default value if this field is not defined.
// +optional
// +kubebuilder:validation:Type=string
// +kubebuilder:validation:Pattern="^([0-9]+(s|m|h))+$"
LeaderElectionLeaseDuration *metav1.Duration `json:"leaderElectionLeaseDuration,omitempty"`

// The interval between attempts by the acting master to renew a leadership slot
// before it stops leading. This must be less than or equal to the lease duration.
// This is only applicable if leader election is enabled.
// Use the klusterlet agent default value if this field is not defined.
// +optional
// +kubebuilder:validation:Type=string
// +kubebuilder:validation:Pattern="^([0-9]+(s|m|h))+$"
LeaderElectionRenewDeadline *metav1.Duration `json:"leaderElectionRenewDeadline,omitempty"`

// The duration the clients should wait between attempting acquisition and renewal
// of a leadership. This is only applicable if leader election is enabled.
// Use the klusterlet agent default value if this field is not defined.
// +optional
// +kubebuilder:validation:Type=string
// +kubebuilder:validation:Pattern="^([0-9]+(s|m|h))+$"
LeaderElectionRetryPeriod *metav1.Duration `json:"leaderElectionRetryPeriod,omitempty"`
}

const (
// ClusterAnnotationsKeyPrefix is the prefix of annotations set on ManagedCluster when creating only.
ClusterAnnotationsKeyPrefix = "agent.open-cluster-management.io"
Expand Down
33 changes: 33 additions & 0 deletions operator/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions operator/v1/zz_generated.swagger_doc_generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading