Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CDI on ARM #3466

Open
3 of 9 tasks
zhlhahaha opened this issue Oct 22, 2024 · 8 comments
Open
3 of 9 tasks

CDI on ARM #3466

zhlhahaha opened this issue Oct 22, 2024 · 8 comments
Labels
kind/enhancement

Comments

@zhlhahaha
Copy link
Contributor

zhlhahaha commented Oct 22, 2024 •
edited
Loading

This is used to record the status of CDI on ARM. We want to make sure CDI works well on Arm64 platforms.
Build Arm64 binary and images

  • native build on Arm64
  • cross build for Arm64
# when doing cross build, export BUILD_ARCH=crossbuild-aarch64 
make bazel-build
make bazel-build-images
make bazel-push-images
make build-functest

There is no error when running the build commands.

  • multi-arch manifest build

Test on Arm64 platform

CI/CD on Arm64

  • integrate one arm64 server in the CI/CD infra
  • create related jobs on Arm64 server

Release Arm64 images

  • Arm64 images releases
  • multi-arch manifest releases
@zhlhahaha
Copy link
Contributor Author

cc: @awels @cfilleke @xpivarc @dhiller

@zhlhahaha
Copy link
Contributor Author

Hi, @awels, I'm currently working on getting CDI to function on Arm and enabling the publication of official multi-arch images. What steps are needed to officially publish multi-arch images?
The e2e tests cover various backends like Ceph, NFS, Istio, and others. To officially publish multi-arch images, is it required to pass all the e2e tests for each backend?

@awels
Copy link
Member

awels commented Oct 23, 2024

No, we just have to note which that certain things are not tested on arm. We have a similar limitation on other arches.

@zhlhahaha
Copy link
Contributor Author

No, we just have to note which that certain things are not tested on arm. We have a similar limitation on other arches.

Ok, thanks, I will record the status here.

@zhlhahaha
Copy link
Contributor Author

zhlhahaha commented Nov 7, 2024 •
edited
Loading

I attempted to run the hpp_classic end-to-end test on the Arm64 platform and encountered the following issues:

  1. The e2e tests in automation/non-csi-hpp.sh rely on the k8s-1.30 provider, which is set up in a nested environment not supported on Arm64. Additionally, the test script isn’t compatible with the kind provider.

  2. After deploying the CDI e2e test environment, two test-related pods fail to start successfully:

  • cdi-file-host
  • imageio-deployment

The cdi-file-host pod displays this error:

nginx: [alert] OPENSSL_init_ssl() failed (SSL: error:80000002:system library::No such file or directory:calling stat(/etc/crypto-policies/back-ends/opensslcnf.config) error:07000075:configuration file routines::ssl command section empty:name=system_default, value=crypto_policy error:0700006D:configuration file routines::module initialization error:module=ssl_conf, value=ssl_module retcode=-1

This issue may be related to the crypto-policies RPM package and requires further investigation.

The imageio-deployment pod uses the image quay.io/kubevirt/fakeovirt:v1.38.0, also limited to x86.

To enable these e2e tests on Arm64, we will need to build multi-arch versions of the image.

@zhlhahaha
Copy link
Contributor Author

Current e2e tests status
FAIL! -- 316 Passed | 79 Failed | 3 Pending | 260 Skipped

Investigating the failure cases now.

Details:

Summarizing 79 Failures:
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [test_id:5081]succeed given a valid token (form)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:817
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Archive upload should [It] succeed given a valid token (alpha)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:402
  [FAIL] Preallocation [test_id:7241] All import paths should contain Preallocation step [It] VddkImport
  /root/go/src/kubevirt.io/containerized-data-importer/tests/import_test.go:1372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:3931]succeed creating import dv with streaming image conversion
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:7202][crit:high][posneg:positive][test_id:8277]succeed creating import dv with custom https headers
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [test_id:5080]succeed given a valid token (async alpha)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:189  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1115][crit:high][test_id:1357]succeed creating import dv with given valid url
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] succeed creating import dv with custom https cert that has a weird filename
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [test_id:5078]succeed given a valid token (async)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:189
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload [It] [test_id:4988]Verify upload to the same pvc fails
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:238
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1115][crit:high][test_id:1379]succeed creating import dv with given valid qcow2 url (https)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1111][crit:high][test_id:1361]succeed creating blank image dv
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1115][crit:high][test_id:1478]succeed creating import dv with given valid registry url
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [test_id:5082]succeed given a valid token (form async)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:817
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Archive upload should [It] fail given an invalid token
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:402
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1115][crit:high][test_id:1379]succeed creating import dv with given valid zst url (https)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:5077]succeed creating import dv from VDDK source
  /root/go/src/kubevirt.io/containe
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload [It] Verify cross-site scripting XSS attempt is escaped accordingly to avoid attack
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:277
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Archive upload should [It] succeed upload of tar.gz
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:402
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:1115][crit:high][test_id:1379]succeed creating import dv with given valid url (https)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:7202][crit:high][posneg:positive][test_id:8278]succeed creating import dv with custom https headers from a secret
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:4334][test_id:6433]succeed creating import dv with given valid registry url and DV barely big enough
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [test_id:1368]succeed given a valid token
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:189
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload should [It] [posneg:negative][test_id:1369]fail given an invalid token
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:189
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Archive upload should [It] succeed upload of tar.xz
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:402
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [rfe_id:XXXX][crit:high][test_id:XXXX]succeed creating import dv from GCS URL using RAW image
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] Namespace with quota [It] [test_id:4981]Should create import pod in namespace with quota
  /root/go/src/kubevirt.io/containerized-data-importer/tests/import_test.go:800
  [FAIL] Namespace with quota [It] [test_id:4983]Should fail to create import pod in namespace with quota, then succeed once the quota is large enough
  /root/go/src/k
  [FAIL] Preallocation Each upload path include preallocation/conversion [It] sync
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1969
  [FAIL] Preallocation Each upload path include preallocation/conversion [It] async
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1969
  [FAIL] Preallocation Each upload path include preallocation/conversion [It] form sync
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:817
  [FAIL] Preallocation Each upload path include preallocation/conversion [It] form async
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:817
  [FAIL] CDIConfig manipulation upload tests [It] [test_id:9063]Should fail upload when TLS profile requires minimal TLS version higher than our client's
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1331
  [FAIL] [rfe_id:1115][crit:high][vendor:[email protected]][level:component]Importer Test Suite [It] [test_id:4969]Should create import pod for blank raw image
  /root/go/src/kubevirt.io/containerized-data-importer/tests/import_test.go:133
  [FAIL] [rfe_id:1115][crit:high][vendor:[email protected]][level:component]Importer Test Suite [It] [test_id:6688] Should retain all multi-stage importer pods after completion with dv annotation cdi.kubevirt.io/storage.pod.retainAfterCompletion=true
  /root/go/src/kubevirt.io/containerized-data-importer/tests/framework/vddk.go:24
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload Verify validation error message on async upload if virtual size > pvc size [It] [test_id:4989]fail given a large virtual size QCOW2 file
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:305
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload Verify validation error message on async upload if virtual size > pvc size [It] fail given a large physical size QCOW2 file
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:305
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload [posneg:negative][test_id:2330]Verify failure on sync upload if virtual size > pvc size [It] fail given a large virtual size RAW XZ file
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:327
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload [posneg:negative][test_id:2330]Verify failure on sync upload if virtual size > pvc size [It] fail given a large virtual size QCOW2 file
  /root/go/s
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:327
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component]Upload tests Standard upload [posneg:negative][test_id:2330]Verify failure on sync upload if virtual size > pvc size [It] fail given a large physical size QCOW2 file
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:327
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [test_id:1354]Should clone data within same namespace [It] with same target size
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [test_id:1354]Should clone data within same namespace [It] with larger target
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [It] [test_id:4953]Should clone imported data within same namespace and preserve fsGroup
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [test_id:1355]Should clone data across different namespaces [It] with same target size
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [test_id:1355]Should clone data across different namespaces [It] with bigger target size
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite HostAssisted Clone [It] [test_id:4954]Should clone data across different namespaces when source initially in use
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests Namespace with quota [It] [test_id:4956]Should create clone in namespace with quota
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests Namespace with quota [It] [test_id:4958]Should fail to clone in namespace with quota when pods have higher requirements, then succeed when quota increased
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests Namespace with quota [It] [test_id:4959]Should create clone in namespace with quota when pods requirements are low enough
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite [It] [test_id
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] all clone tests [rfe_id:1277][crit:high][vendor:[email protected]][level:component]Cloner Test Suite [It] [test_id:4278] Clone datavolume with long name including special character '.'
  /root/go/src/kubevirt.io/containerized-data-importer/tests/cloner_test.go:2967
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests [It] Upload an image exactly the same size as DV request (bz#2064936)
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1440
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests DataVolume Garbage Collection Should [It] [test_id:8566] garbage collect dvs after completion when TTL is 0
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1525
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests DataVolume Garbage Collection Should [It] [test_id:8570] Add DeleteAfterCompletion annotation to a legacy DV
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1525
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests [It] [test_id:3993] Upload image to data volume and verify retry count
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1581
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests Upload datavolume creates correct scratch space, pod and service names [It] [test_id:4273] with short DataVolume name
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1696
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests Upload datavolume creates correct scratch space, pod and service names [It] [test_id:4274] with long DataVolume name
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1696
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests Upload datavolume creates correct scratch space, pod and service names [It] [test_id:4275] with long DataVolume name including special chars '.'
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1696
  [FAIL] [rfe_id:138][crit:high][vendor:[email protected]][level:component] Upload tests [It] [test_id:1985] Upload datavolume should succeed on retry after failure
  /root/go/src/kubevirt.io/containerized-data-importer/tests/upload_test.go:1764
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:3932]succeed creating dv from imageio source
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:3945]succeed creating dv from imageio source that does not support extents query
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:372
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:5079]should fail with "AwaitingVDDK" reason when VDDK image config map is not present
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:147
  [FAIL] [vendor:[email protected]][level:component]DataVolume tests Verify DataVolume should [It] [test_id:5080]succeed importing VDDK data volume with init image URL set
  /root/go/src/kubevirt.io/containerized-data-importer/tests/datavolume_test.go:147
  [FAIL] DataImportCron should [It] [test_id:7403] succeed importing initial PVC from registry URL
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:124
  [FAIL] DataImportCron should [It] [test_id:7414] succeed importing PVC from registry URL on source digest update
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:124
  [FAIL] DataImportCron should [It] [test_id:8266] succeed deleting error DVs when importing new ones
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:124
  [FAIL] DataImportCron [It] [test_id:10040] Should get digest updated by external poller
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:111
  [FAIL] DataImportCron [It] [test_id:10360] Should allow an empty schedule to trigger an external update to the source
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:111
  [FAIL] DataImportCron Succeed garbage collecting sources when importing new ones [It] [test_id:7406] with PVC & DV sources
  /root/go/src/kubevirt.io/containerized-data-importer/tests/dataimportcron_test.go:124Ran 395 of 658 Specs in 15013.957 seconds
FAIL! -- 316 Passed | 79 Failed | 3 Pending | 260 Skipped

@zhlhahaha
Copy link
Contributor Author

Summary of the failures:

30 upload_test.go 
16 datavolume_test.go 
11 cloner_test.go 
6 dataimportcron_test.go
3 import_test.go
1 vddk.go

@zhlhahaha
Copy link
Contributor Author

Hi @awels , I am trying to figure out the reasons of e2e test failures.
For the upload test, I am trying to verify the upload function of CDI manually. Here is the guidance I followed https://github.com/kubevirt/containerized-data-importer/blob/main/doc/upload.md

However, I got an issue when trying to upload the datavolume token by running the following command:

kubectl apply -f manifests/example/upload-datavolume-token.yaml -o yaml

It shows that

error: resource mapping not found for name: "upload-datavolume" namespace: "default" from "manifests/example/upload-datavolume-token.yaml": no matches for kind "UploadTokenRequest" in version "upload.cdi.kubevirt.io/v1beta1"
ensure CRDs are installed first

Do you have any ideas or suggestions?

Here is the information about the cluster:
The k8s cluster is created via the kind provider,
and setup CDI by running following command

export TARGET=kind-1.28
export KUBEVIRT_STORAGE=hpp
export HPP_CLASSIC=true
export CDI_E2E_SKIP=Destructive
make cluster-sync

Now all pods running well

# kubectl get pods -A
NAMESPACE              NAME                                              READY   STATUS    RESTARTS     AGE
cdi                    cdi-apiserver-d7f8c64f5-n6vw4                     1/1     Running   2 (4d ago)   4d
cdi                    cdi-bad-webserver-794d84ccc7-l49gb                1/1     Running   0            4d
cdi                    cdi-deployment-78588d6c74-ggw6z                   1/1     Running   0            4d
cdi                    cdi-docker-registry-host-7587f89474-2b9tn         3/3     Running   0            4d
cdi                    cdi-file-host-5b7c89c45f-59rhv                    2/2     Running   0            4d
cdi                    cdi-operator-777c4fc799-4h8b7                     1/1     Running   0            4d
cdi                    cdi-sample-populator-78bc6767b6-2bk9d             1/1     Running   0            4d
cdi                    cdi-test-proxy-6db4d74bd4-csf8t                   1/1     Running   0            4d
cdi                    cdi-uploadproxy-67959b544c-r8zpb                  1/1     Running   0            4d
cdi                    imageio-deployment-9b955cc95-2zrbl                2/2     Running   0            4d
cdi                    vcenter-deployment-6855b8cff9-xfj4g               1/1     Running   0            4d
cert-manager           cert-manager-744cd64f59-jb5fr                     1/1     Running   0            4d
cert-manager           cert-manager-cainjector-65848d86bb-bnr2n          1/1     Running   0            4d
cert-manager           cert-manager-webhook-69977f4975-6gkfs             1/1     Running   0            4d
hostpath-provisioner   hostpath-provisioner-csi-6q4jk                    4/4     Running   0            4d
hostpath-provisioner   hostpath-provisioner-lcsdr                        1/1     Running   0            4d
hostpath-provisioner   hostpath-provisioner-operator-64fbc44d54-26k8z    1/1     Running   0            4d
kube-system            coredns-5dd5756b68-77pjg                          1/1     Running   0            4d1h
kube-system            coredns-5dd5756b68-qdhf6                          1/1     Running   0            4d1h
kube-system            etcd-kind-1.28-control-plane                      1/1     Running   0            4d1h
kube-system            kindnet-4smhq                                     1/1     Running   0            4d1h
kube-system            kube-apiserver-kind-1.28-control-plane            1/1     Running   0            4d1h
kube-system            kube-controller-manager-kind-1.28-control-plane   1/1     Running   0            4d1h
kube-system            kube-proxy-8pf6z                                  1/1     Running   0            4d1h
kube-system            kube-scheduler-kind-1.28-control-plane            1/1     Running   0            4d1h
local-path-storage     local-path-provisioner-6f8956fb48-58psr           1/1     Running   0            4d1h

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@awels @zhlhahaha