Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNM Checking CRC Cloud with image based on OCP 4.16 #2019

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

danpawlik
Copy link
Contributor

This is early test PR that uses new way for deploying the CRC - crc-cloud [1] and image that is base on OCP 4.16 rc4 with small modifications [2]. In the future, that image would be created by the nodepool-builder, but right now it is just a PoC (Proof of concept). Also is worth to mention, that the base job includes few workarounds until the PR in the official projects are not merged [3][4][5]

[1] https://github.com/crc-org/crc-cloud
[2] https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/31853
[3] crc-org/crc-cloud#188
[4] crc-org/crc-cloud#189
[5] crc-org/crc-cloud#190

@danpawlik danpawlik requested a review from bshewale July 3, 2024 11:35
@danpawlik danpawlik requested review from lewisdenny and removed request for bshewale and lewisdenny July 3, 2024 11:35
Copy link
Contributor

openshift-ci bot commented Jul 3, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@danpawlik danpawlik requested review from viroel and bshewale July 3, 2024 11:35
Copy link

Zuul encountered a syntax error while parsing its
configuration in the repo openstack-k8s-operators/ci-framework on branch main. The
problem was:

found duplicate anchor; first occurrence
in "openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main", line 130, column 24
second occurrence
in "openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main", line 319, column 24

@danpawlik danpawlik requested a review from lewisdenny July 3, 2024 11:35
@danpawlik danpawlik force-pushed the crc-cloud branch 3 times, most recently from 473ebf5 to d1c7db4 Compare July 3, 2024 11:43
@danpawlik
Copy link
Contributor Author

recheck

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/5df149fbb3624440805861c355c8f06f

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 10m 35s
podified-multinode-edpm-deployment-crc RETRY_LIMIT Host unreachable in 42m 36s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 14m 46s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 1h 36m 01s
cifmw-multinode-tempest RETRY_LIMIT Host unreachable in 1h 29m 24s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 59s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 42s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 54s
cifmw-multinode-kuttl RETRY_LIMIT Host unreachable in 14m 50s

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/178dd4b0c06f43379074b294026ce371

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 37m 13s
podified-multinode-edpm-deployment-crc RETRY_LIMIT Host unreachable in 1h 28m 51s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 19m 33s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 1h 34m 41s
cifmw-multinode-tempest RETRY_LIMIT Host unreachable in 1h 28m 17s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 39s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 54s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 49s
cifmw-multinode-kuttl RETRY_LIMIT Host unreachable in 1h 27m 05s

@danpawlik
Copy link
Contributor Author

recheck

1 similar comment
@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/b5475b5f45ac4cea960f14d2261e0cab

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 23m 39s
podified-multinode-edpm-deployment-crc FAILURE in 1h 13m 38s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 25m 40s
podified-multinode-hci-deployment-crc FAILURE in 1h 50m 23s
cifmw-multinode-tempest FAILURE in 1h 16m 44s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 48s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 19s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 08s
cifmw-multinode-kuttl FAILURE in 45m 09s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/0b6944a2529044bc9017f6e66edb5ed9

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 02m 17s
podified-multinode-edpm-deployment-crc FAILURE in 1h 13m 27s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 27m 08s
podified-multinode-hci-deployment-crc FAILURE in 1h 47m 44s
cifmw-multinode-tempest FAILURE in 1h 18m 09s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 01s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 02s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 19s
cifmw-multinode-kuttl FAILURE in 1h 05m 08s

@danpawlik danpawlik requested review from rlandy and elfiesmelfie July 5, 2024 12:21
- name: compute-0
label: cloud-centos-9-stream-tripleo
- name: crc
label: crc-cloud-ocp-4-16-0-rc4-3xl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually, all I need is the nodeset. I'll upload a PR for STF with just that change.

In the meantime, I will need to check what the base-crc-cloud job does and how it differs from base extracted crc.

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/a0054c0507f04b25979eae7feb64bb7c

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 30m 11s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 9m 12s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 14m 42s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 14m 43s
cifmw-multinode-tempest RETRY_LIMIT in 8m 57s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 49s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 03s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 04s
cifmw-multinode-kuttl RETRY_LIMIT in 7m 31s

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/e619e437b2b646c9a8339ffb0f5c7376

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 19m 45s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 23s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 24m 45s
podified-multinode-hci-deployment-crc FAILURE in 22m 24s
cifmw-multinode-tempest RETRY_LIMIT in 4m 22s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 21s
cifmw-pod-pre-commit FAILURE in 7m 41s
cifmw-pod-zuul-files FAILURE in 4m 16s
cifmw-multinode-kuttl TIMED_OUT in 2h 10m 08s
✔️ build-push-container-cifmw-client SUCCESS in 36m 22s

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/fea2419f3b3646b99618208091bcf027

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 46m 18s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 3m 59s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 33m 00s
podified-multinode-hci-deployment-crc FAILURE in 16m 48s
cifmw-multinode-tempest RETRY_LIMIT in 8m 54s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 16s
cifmw-pod-pre-commit FAILURE in 6m 57s
cifmw-pod-zuul-files FAILURE in 5m 14s
cifmw-multinode-kuttl FAILURE in 1h 16m 35s
✔️ build-push-container-cifmw-client SUCCESS in 37m 28s

Copy link

github-actions bot commented Nov 3, 2024

This PR is stale because it has been for over 15 days with no activity.
Remove stale label or comment or this will be closed in 7 days.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/f6dd1f9fda4a4054a7012e04e1e2d547

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 10m 34s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 07s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 26m 39s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 14m 39s
cifmw-multinode-tempest RETRY_LIMIT in 4m 15s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 46s
cifmw-pod-pre-commit FAILURE in 7m 28s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 36s
cifmw-multinode-kuttl RETRY_LIMIT in 9m 08s
✔️ build-push-container-cifmw-client SUCCESS in 21m 42s

@danpawlik
Copy link
Contributor Author

recheck

1 similar comment
@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/7d41647bca8641169d5dca36a76ede4d

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 46m 47s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 22s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 25s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 14m 36s
cifmw-multinode-tempest RETRY_LIMIT in 4m 20s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 25s
cifmw-pod-pre-commit FAILURE in 7m 26s
✔️ cifmw-pod-zuul-files SUCCESS in 3m 50s
cifmw-multinode-kuttl RETRY_LIMIT in 9m 01s
✔️ build-push-container-cifmw-client SUCCESS in 21m 08s

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3ddae3f2b32546f6a94d08102f77b2ee

✔️ openstack-k8s-operators-content-provider SUCCESS in 41m 56s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 38s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 12m 06s
podified-multinode-hci-deployment-crc FAILURE in 15m 53s
cifmw-multinode-tempest RETRY_LIMIT in 4m 47s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 27s
cifmw-pod-pre-commit FAILURE in 7m 19s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 03s
cifmw-multinode-kuttl FAILURE in 24m 42s
✔️ build-push-container-cifmw-client SUCCESS in 20m 45s

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/be401cb0d4344842a27290bfe17e2a1d

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 34m 50s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 05s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 22m 36s
podified-multinode-hci-deployment-crc FAILURE in 14m 49s
cifmw-multinode-tempest RETRY_LIMIT in 4m 20s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 6m 49s
cifmw-pod-pre-commit FAILURE in 7m 10s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 16s
cifmw-multinode-kuttl FAILURE in 21m 39s
✔️ build-push-container-cifmw-client SUCCESS in 21m 59s

Copy link

This PR is stale because it has been for over 15 days with no activity.
Remove stale label or comment or this will be closed in 7 days.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/b4501d014f844f0186688401230512ae

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 38m 46s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 5m 17s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 26m 42s
podified-multinode-hci-deployment-crc FAILURE in 17m 14s
cifmw-multinode-tempest RETRY_LIMIT in 5m 09s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 46s
cifmw-pod-pre-commit FAILURE in 7m 02s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 39s
cifmw-multinode-kuttl FAILURE in 26m 44s
✔️ build-push-container-cifmw-client SUCCESS in 21m 00s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5533b78b22144c7091429b69b077b171

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 27m 07s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 17s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 14m 02s
podified-multinode-hci-deployment-crc RETRY_LIMIT in 3h 04m 16s
cifmw-multinode-tempest RETRY_LIMIT in 4m 18s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 03s
cifmw-pod-pre-commit FAILURE in 7m 08s
✔️ cifmw-pod-zuul-files SUCCESS in 3m 57s
cifmw-multinode-kuttl RETRY_LIMIT in 2h 31m 45s
✔️ build-push-container-cifmw-client SUCCESS in 21m 20s

@danpawlik
Copy link
Contributor Author

danpawlik commented Dec 13, 2024

@lewisdenny temporary we need to wait for new image.
Root cause that it stuck in base job:

sudo crictl ps -a | grep kube-apiserver-crc

(...)
72377fd6eee38       be29b56d0cf815ae8ca03f8a4154f537d543eb0a912717ef3b68a5cdbbf74bea   10 minutes ago       Running             kube-apiserver                                0                   d98dba5d0d05c   kube-apiserver-crc

sudo crictl logs 72377fd6eee38

(...)
E1213 15:42:06.049609      12 authentication.go:73] "Unable to authenticate the request" err="[x509: certificate has expired or is not yet valid: current time 2024-12-13T15:42:06Z is after 2024-12-04T11:38:0      4Z, verifying certificate SN=8043119799285590504976287006898681274, SKID=, AKID=2E:D1:46:37:D3:74:D8:8A:BD:06:50:90:0E:4C:34:AA:D5:23:68:31 failed: x509: certificate has expired or is not yet valid: current       time 2024-12-13T15:42:06Z is after 2024-12-04T11:38:04Z]"


[core@crc kubelet]$ sudo openssl x509 -noout -dates -in /var/lib/kubelet/pki/kubelet-server-current.pem
notBefore=Nov  4 11:42:36 2024 GMT
notAfter=Dec  4 11:38:04 2024 GMT

I will search if we can regen the cert, but new CRC image should be released on Monday, so maybe it is not worth to check (how to do it)

This is early test PR that uses new way for deploying the CRC -
crc-cloud [1] and image that is base on OCP 4.18.
with small modifications [2]. In the future, that image would be created
by the nodepool-builder, but right now it is just a PoC (Proof of concept).
Also is worth to mention, that the base job includes few workarounds
until the PR in the official projects are not merged [3][4][5]

[1] https://github.com/crc-org/crc-cloud
[2] https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/31853
[3] crc-org/crc-cloud#188
[4] crc-org/crc-cloud#189
[5] crc-org/crc-cloud#190
Copy link
Contributor

openshift-ci bot commented Dec 16, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cescgina for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@danpawlik
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/05ca0e1cc31142d4a35d33d22ef0c6a7

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 40m 33s
podified-multinode-edpm-deployment-crc RETRY_LIMIT in 4m 17s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 09s
podified-multinode-hci-deployment-crc FAILURE in 16m 19s
cifmw-multinode-tempest RETRY_LIMIT in 4m 23s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 42s
cifmw-pod-pre-commit FAILURE in 7m 29s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 09s
cifmw-multinode-kuttl FAILURE in 24m 44s
✔️ build-push-container-cifmw-client SUCCESS in 20m 44s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants