Skip to content

Commit

Permalink
Support complex node affinity in cluster spec
Browse files Browse the repository at this point in the history 
Use the nodeAffinity, like one would set it on pods to allow more complex
deployment scenarios. We need to respect these settings when creating
Satellites, but also when creating our own deployments.

For consistency, the same options are also available on sht satellite
configuration resources.

Signed-off-by: Moritz "WanzenBug" Wanzenböck <[email protected]>
  • Loading branch information
@WanzenBug
WanzenBug committed Nov 17, 2023
1 parent 3009d0b commit 0f3b0bc
Show file tree
Hide file tree
Showing 25 changed files with 822 additions and 19 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/checks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.21'
- uses: golangci/golangci-lint-action@v3
with:
args: --timeout=3m
Expand All @@ -29,7 +29,7 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.21'
- uses: actions/setup-python@v4
- name: Run pre-commit checks on changes files
uses: pre-commit/[email protected]
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add image configuration for CSI sidecars.
- Check kernel module parameters for DRBD on load.
- Automatically set SELinux labels when loading kernel modules.
- Allow more complex node selection by adding `LinstorCluster.spec.nodeAffinity`.

### Changed

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.4
# Build the manager binary
FROM --platform=$BUILDPLATFORM golang:1.20 as builder
FROM --platform=$BUILDPLATFORM golang:1.21 as builder

WORKDIR /workspace
# Copy the Go Modules manifests
Expand Down
6 changes: 6 additions & 0 deletions api/v1/linstorcluster_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package v1

import (
cmmetav1 "github.com/cert-manager/cert-manager/pkg/apis/meta/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

Expand All @@ -38,6 +39,11 @@ type LinstorClusterSpec struct {
// +kubebuilder:validation:Optional
NodeSelector map[string]string `json:"nodeSelector,omitempty"`

// NodeAffinity selects the nodes on which LINSTOR Satellite will be deployed.
// See https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
// +kubebuilder:validation:Optional
NodeAffinity *corev1.NodeSelector `json:"nodeAffinity,omitempty"`

// Properties to apply on the cluster level.
//
// Use to create default settings for DRBD that should apply to all resources or to configure some other cluster
Expand Down
6 changes: 6 additions & 0 deletions api/v1/linstorsatelliteconfiguration_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ limitations under the License.
package v1

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

Expand All @@ -30,6 +31,11 @@ type LinstorSatelliteConfigurationSpec struct {
// +kubebuilder:validation:Optional
NodeSelector map[string]string `json:"nodeSelector,omitempty"`

// NodeAffinity selects which LinstorSatellite resources this spec should be applied to.
// See https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
// +kubebuilder:validation:Optional
NodeAffinity *corev1.NodeSelector `json:"nodeAffinity,omitempty"`

// Patches is a list of kustomize patches to apply.
//
// See https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/patches/ for how to create patches.
Expand Down
11 changes: 11 additions & 0 deletions api/v1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

170 changes: 170 additions & 0 deletions charts/piraeus/templates/crds.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,91 @@ spec:
for accessing remotes for backups. See https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-encrypt_commands
for more information."
type: string
nodeAffinity:
description: NodeAffinity selects the nodes on which LINSTOR Satellite
will be deployed. See https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
properties:
nodeSelectorTerms:
description: Required. A list of node selector terms. The terms
are ORed.
items:
description: A null or empty node selector term matches no objects.
The requirements of them are ANDed. The TopologySelectorTerm
type implements a subset of the NodeSelectorTerm.
properties:
matchExpressions:
description: A list of node selector requirements by node's
labels.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchFields:
description: A list of node selector requirements by node's
fields.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
x-kubernetes-map-type: atomic
type: array
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
nodeSelector:
additionalProperties:
type: string
Expand Down Expand Up @@ -569,6 +654,91 @@ spec:
connections on behalf of DRBD."
type: boolean
type: object
nodeAffinity:
description: NodeAffinity selects which LinstorSatellite resources
this spec should be applied to. See https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
properties:
nodeSelectorTerms:
description: Required. A list of node selector terms. The terms
are ORed.
items:
description: A null or empty node selector term matches no objects.
The requirements of them are ANDed. The TopologySelectorTerm
type implements a subset of the NodeSelectorTerm.
properties:
matchExpressions:
description: A list of node selector requirements by node's
labels.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchFields:
description: A list of node selector requirements by node's
fields.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
x-kubernetes-map-type: atomic
type: array
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
nodeSelector:
additionalProperties:
type: string
Expand Down
85 changes: 85 additions & 0 deletions config/crd/bases/piraeus.io_linstorclusters.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,91 @@ spec:
for accessing remotes for backups. See https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-encrypt_commands
for more information."
type: string
nodeAffinity:
description: NodeAffinity selects the nodes on which LINSTOR Satellite
will be deployed. See https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
properties:
nodeSelectorTerms:
description: Required. A list of node selector terms. The terms
are ORed.
items:
description: A null or empty node selector term matches no objects.
The requirements of them are ANDed. The TopologySelectorTerm
type implements a subset of the NodeSelectorTerm.
properties:
matchExpressions:
description: A list of node selector requirements by node's
labels.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchFields:
description: A list of node selector requirements by node's
fields.
items:
description: A node selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: Represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists,
DoesNotExist. Gt, and Lt.
type: string
values:
description: An array of string values. If the operator
is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. If the operator is Gt or Lt,
the values array must have a single element, which
will be interpreted as an integer. This array is
replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
x-kubernetes-map-type: atomic
type: array
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
nodeSelector:
additionalProperties:
type: string
Expand Down
Loading

0 comments on commit 0f3b0bc

Please sign in to comment.