forked from vmware-tanzu/velero
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add design for Extending VolumePolicies to support more actions
Signed-off-by: Shubham Pampattiwar <[email protected]> add changelog Signed-off-by: Shubham Pampattiwar <[email protected]> fix codespell Signed-off-by: Shubham Pampattiwar <[email protected]> update codeblocks for language syntax rendering Signed-off-by: Shubham Pampattiwar <[email protected]>
- Loading branch information
1 parent
ad114f8
commit a11ff09
Showing
2 changed files
with
162 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Add design for Extending VolumePolicies to support more actions |
161 changes: 161 additions & 0 deletions
161
design/Extend-VolumePolicies-to-support-more-actions.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| # Extend VolumePolicies to support more actions | ||
|
|
||
| ## Abstract | ||
|
|
||
| Currently, the [VolumePolicies feature](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md) which can be used to filter/handle volumes during backup only supports the skip action on matching conditions. Users need more actions to be supported. | ||
|
|
||
| ## Background | ||
|
|
||
| The `VolumePolicies` feature was introduced in Velero 1.11 as a flexible way to handle volumes. The main agenda of introducing the VolumePolicies feature was to improve the overall user experience when performing backup operations for volume resources, the feature enables users to group volumes according the `conditions` (criteria) specified and also lets you specify the `action` that velero needs to take for these grouped volumes during the backup operation. The limitation being that currently `VolumePolicies` only supports `skip` as an action, We want to extend the `action` functionality to support more usable options like `fs-backup-volumes` (File system backup) and `fs-backup-volumes-excludes` (VolumeSnapshots). | ||
|
|
||
| ## Goals | ||
| - Extending the VolumePolicies to support more actions like file system backup(`fs-backup-volumes`) and snapshots(`fs-backup-volumes-excludes`). | ||
| - Improve user experience when backing up Volumes via Velero | ||
|
|
||
| ## Non-Goals | ||
| - No changes to existing approaches to opt-in/opt-out annotations for volumes | ||
| - No changes to existing `VolumePolicies` functionalities | ||
|
|
||
|
|
||
| ## Use-cases/Scenarios | ||
|
|
||
| **Use-case 1:** | ||
| - A user wants to use `fs-backup-volumes-excludes` (volumesnapshots) backup option for all the csi supported volumes and `fs-backup-volumes` for the rest of the volumes. | ||
| - Currently, velero supports this use-case but the user experience is not that great. | ||
| - The user will have to individually annotate the volume mounting pod with the annotation "backup.velero.io/backup-volumes" for `fs-backup-volumes` | ||
| - This becomes cumbersome at scale. | ||
| - Using `VolumePolicies`, the user can just specify a simple `VolumePolicy` like for csi supported volumes and rest can be defaulted to `fs-backup-volumes`: | ||
| ```yaml | ||
| version: v1 | ||
| volumePolicies: | ||
| - conditions: | ||
| csi: | ||
| driver: aws.efs.csi.driver | ||
| action: | ||
| type: fs-backup-volumes-excludes | ||
| ``` | ||
| **Use-case 2:** | ||
| - A user wants to use `fs-backup-volumes` for nfs volumes | ||
| - In such a scenario the user can just specify a `VolumePolicy` like: | ||
| ```yaml | ||
| version: v1 | ||
| volumePolicies: | ||
| - conditions: | ||
| nfs: | ||
| server: 192.168.200.90 | ||
| action: | ||
| type: fs-backup-volumes | ||
| ``` | ||
| ## High-Level Design | ||
| - When the VolumePolicy action is set as `fs-backup-volumes` the backup workflow modifications would be: | ||
| - We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L93) on all the items that are to be backed up | ||
| - Here when we encounter [Pods as an item ](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L193)we segregate volume to be backed up using `fs-backup-volumes` and `fs-backup-volumes-excludes` | ||
| - We will have to modify the backup workflow to account for the `fs-backup-volumes` VolumePolicy action | ||
| - The [`GetVolumesByPod()`](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L204) function signature will be modified to take in to account the VolumePolicies specified by the user. | ||
| - The `includedVolumes` list returned by the `GetVolumesByPod` function will include the volumes matching the `fs-backup-volumes` action conditions in VolumePolicy. These volumes will be backed up via the file system backup workflow. | ||
| - When the VolumePolicy action is set as `fs-backup-volumes-excludes` the backup workflow modifications would be: | ||
| - Once again, We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L93) on all the items that are to be backed up | ||
| - Here when we encounter [Pods as an item](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L193) we segregate volume to be backed up using `fs-backup-volumes` and `fs-backup-volumes-excludes` | ||
| - We will have to modify the backup workflow to account for the `fs-backup-volume-excludes` VolumePolicy action | ||
| - The [`GetVolumesByPod()`](https://github.com/vmware-tanzu/velero/blob/7f73acab1644f7170779b7225a5d7336b20f0868/pkg/backup/item_backupper.go#L204) function signature will be modified to take in to account the VolumePolicies specified by the user. | ||
| - The `optedOutVolumes` list returned by the `GetVolumesByPod` function will include the volumes matching the `fs-backup-volume-exclude` action conditions in VolumePolicy. These volumes will be backed up via the snapshots workflow. | ||
|
|
||
|
|
||
| ## Detailed Design | ||
| - Update VolumePolicy action type validation to account for `fs-backup-volumes` and `fs-backup-volumes-exclude` as valid VolumePolicy actions. | ||
| - Modifications needed for `fs-backup-volumes` action: | ||
| - In `GetVolumesByPod` function, process for `VolumePolicy` and check for volume matching `fs-backup-volumes` action. | ||
| - If true, then append the volume to `podVolumes` list, so that fs backup workflow is executed for the volume | ||
| - Make sure to take a union of both the approaches (Annotation Opt-in + `fs-backup-volumes` volume policy action) and remove duplicated from `podVolumes` list | ||
|
|
||
| - Modifications needed for `fs-backup-volumes-exclude` action: | ||
| - In `GetVolumesByPod` function, process for `VolumePolicy` and check for volume matching `fs-backup-volumes-excludes` action. | ||
| - If true, then append the volume to `optedOutVolumes` list, so that volume-snapshot backup workflow is executed for the volume | ||
| - Make sure to take a union of both the approaches (Annotation Opt-out + `fs-backup-volumes-excludes` volume policy action) and remove duplicated from `optedOutVolumes` list | ||
|
|
||
|
|
||
| ## Implementation | ||
|
|
||
| - The implementation should be included in velero 1.13 | ||
|
|
||
| - Updated `GetVolumesByPod` function signature will look like | ||
| ```go | ||
| func GetVolumesByPod(pod *corev1api.Pod, defaultVolumesToFsBackup bool, volumePolicies *resourcepolicies.Policies) ([]string, []string, error) | ||
| ``` | ||
|
|
||
| - List tracking `optedOutVolumes` and `podVolumes` | ||
| ```go | ||
| // tracks the volumes that have been explicitly opted out of backup via the annotation in the pod and also via `fs-backup-volumes-exclude` volume policy action | ||
| optedOutVolumes := make([]string, 0) | ||
|
|
||
| // tracks the volumes that have been explicitly opted in for fs backup via the annotation in the pod and also via `fs-backup-volumes` volume policy action | ||
| podVolumes := []string{} | ||
| ``` | ||
|
|
||
| - The processing of VolumePolicies in `GetVolumesByPod` function will look somewhat similar to the following: | ||
|
|
||
| ```go | ||
| //process volume policy actions | ||
| if volumePolicies != nil { | ||
| // Now for each of the Volumes check for action | ||
| for _, vol := range pod.Spec.Volumes { | ||
| action, err := volumePolicies.GetMatchAction(vol) | ||
| if err != nil { | ||
| return nil, nil, err | ||
| } | ||
| // Now if the matched action is `fs-backup-exclude` then add that Volume to the optedOutVolumes List | ||
| if action != nil && action.Type == resourcepolicies.FSBackupVolumesExcludes { | ||
| // append volume to optedOutVolumes list | ||
| optedOutVolumes = append(optedOutVolumes, vol.Name) | ||
| } | ||
|
|
||
| // Now if the matched action is `fs-backup-volume` then add that Volume to the podVolumes List | ||
| if action != nil && action.Type == resourcepolicies.FSBackupVolumes { | ||
| // append volume to podVolumes list | ||
| podVolumes = append(podVolumes, vol.Name) | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| - Taking union of annotations and VolumePolicy approaches and update the list accordingly: | ||
|
|
||
| ```go | ||
| // Now take union of annotations opt-in approach volume and fs-backup-volume volume policy action | ||
| OptInAnnotatedPodVolumes := GetVolumesToBackup(pod) | ||
| for _, optedInAnnotatedVolume := range OptInAnnotatedPodVolumes { | ||
| volAlreadyIncluded := false | ||
| for _, podVolume := range podVolumes { | ||
| if podVolume == optedInAnnotatedVolume { | ||
| volAlreadyIncluded = true | ||
| } | ||
| } | ||
| if !volAlreadyIncluded { | ||
| podVolumes = append(podVolumes, optedInAnnotatedVolume) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ```go | ||
| // Now take union of annotations opt-out approach volume and fs-backup-volume-exclude volume policy action | ||
| OptedOutAnnotatedVolumes := getVolumesToExclude(pod) | ||
| for _, OptedOutAnnotatedVolume := range OptedOutAnnotatedVolumes { | ||
| volAlreadyIncludedInoptedOutVolumes := false | ||
| for _, vol := range optedOutVolumes { | ||
| if vol == OptedOutAnnotatedVolume { | ||
| volAlreadyIncludedInoptedOutVolumes = true | ||
| } | ||
| } | ||
| if !volAlreadyIncludedInoptedOutVolumes { | ||
| optedOutVolumes = append(optedOutVolumes, OptedOutAnnotatedVolume) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Related to Design | ||
|
|
||
| [Handle backup of volumes by resources filters](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md) | ||
|
|
||
| ## Alternatives Considered | ||
| Same as the earlier design as this is an extension of the original VolumePolicies design |