Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test/e2e: docker: Fix KBS test that doesn't compile #2000

Conversation

stevenhorsman
Copy link
Member

As part of the work in #1935, EnableKbsCustomizedPolicy was replaced with EnableKbsCustomizedResourcePolicy and EnableKbsCustomizedAttestationPolicy, but the docker tests wasn't updated, so it doesn't compile, so fix this

@stevenhorsman
Copy link
Member Author

stevenhorsman commented Aug 12, 2024

This is failing the tests for me at the moment:

time="2024-08-12T03:31:40-07:00" level=info msg="Installing peerpod-ctrl"
time="2024-08-12T03:31:43-07:00" level=info msg="Wait for the peerpod-ctrl deployment to be available"
time="2024-08-12T03:31:48-07:00" level=info msg="Creating namespace 'coco-pp-e2e-test-dc187d6f'..."
time="2024-08-12T03:31:48-07:00" level=info msg="Wait for namespace 'coco-pp-e2e-test-dc187d6f' be ready..."
time="2024-08-12T03:31:53-07:00" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-dc187d6f'..."
time="2024-08-12T03:31:53-07:00" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-dc187d6f' is ready for use"
=== RUN   TestDockerKbsKeyRelease
time="2024-08-12T03:31:53-07:00" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/deny_all.rego"
=== PAUSE TestDockerKbsKeyRelease
=== CONT  TestDockerKbsKeyRelease
    common_suite.go:604: Do test kbs key release failure case
=== RUN   TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:265: Waiting for containers in pod: kbs-failure are ready
=== RUN   TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
=== NAME  TestDockerKbsKeyRelease
    common_suite.go:623: PASS as failed to access key.bin:
=== NAME  TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
    assessment_runner.go:421: Output when execute test commands:
=== NAME  TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:607: Deleting pod kbs-failure...
    assessment_runner.go:614: Pod kbs-failure has been successfully deleted within 60s
time="2024-08-12T03:32:19-07:00" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/allow_all.rego"
=== NAME  TestDockerKbsKeyRelease
    common_suite.go:580: Do test kbs key release
=== RUN   TestDockerKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: kbs-key-release are ready
=== RUN   TestDockerKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
    assessment_runner.go:421: Output when execute test commands: command terminated with exit code 1
    assessment_runner.go:423: command terminated with exit code 1
=== NAME  TestDockerKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:607: Deleting pod kbs-key-release...
    assessment_runner.go:614: Pod kbs-key-release has been successfully deleted within 60s
--- FAIL: TestDockerKbsKeyRelease (50.70s)
    --- PASS: TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test (25.25s)
        --- PASS: TestDockerKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed (5.17s)
    --- FAIL: TestDockerKbsKeyRelease/KbsKeyReleasePod_test (25.22s)
        --- FAIL: TestDockerKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.16s)

The KBS log shows that no get requests were made during this testing:

$ kubectl logs deployment/kbs -n coco-tenant
Defaulted container "kbs" out of: kbs, copy-config (init)
[2024-08-10T03:15:14Z INFO  kbs] Using config file /etc/kbs/kbs-config.toml
[2024-08-10T03:15:14Z WARN  attestation_service::rvps] No RVPS address provided and will launch a built-in rvps
[2024-08-10T03:15:14Z INFO  attestation_service::token::simple] No Token Signer key in config file, create an ephemeral key and without CA pubkey cert
[2024-08-10T03:15:14Z INFO  kbs] Starting HTTP server at [0.0.0.0:8080]
[2024-08-10T03:15:14Z INFO  actix_server::builder] starting 4 workers
[2024-08-10T03:15:14Z INFO  actix_server::server] Tokio runtime found; starting in existing Tokio runtime
[2024-08-12T10:31:53Z INFO  actix_web::middleware::logger] 172.18.0.3 "POST /kbs/v0/resource-policy HTTP/1.1" 200 0 "-" "kbs-client/0.1.0" 0.001637
[2024-08-12T10:32:19Z INFO  actix_web::middleware::logger] 172.18.0.3 "POST /kbs/v0/resource-policy HTTP/1.1" 200 0 "-" "kbs-client/0.1.0" 0.001156
[2024-08-12T11:05:50Z ERROR kbs::http::error] Set secret failed: write local fs
[2024-08-12T11:05:50Z INFO  actix_web::middleware::logger] 172.18.0.3 "POST /kbs/v0/resource/reponame/workload_key/key.bin HTTP/1.1" 401 125 "-" "kbs-client/0.1.0" 0.002470
[2024-08-12T11:05:51Z INFO  actix_web::middleware::logger] 172.18.0.3 "POST /kbs/v0/resource-policy HTTP/1.1" 200 0 "-" "kbs-client/0.1.0" 0.001923
[2024-08-12T11:06:16Z INFO  actix_web::middleware::logger] 172.18.0.3 "POST /kbs/v0/resource-policy HTTP/1.1" 200 0 "-" "kbs-client/0.1.0" 0.001704

there is an error in the set secret though: Set secret failed: write local fs?

I'll try debugging this further, unless @liudalibj you've already looked into this/can spot that EnableKbsCustomizedResourcePolicy wasn't the correct replacement?

@stevenhorsman stevenhorsman force-pushed the docker-provider-kbs-fix branch from b0cbe27 to 223bc8e Compare August 12, 2024 11:02
@stevenhorsman
Copy link
Member Author

stevenhorsman commented Aug 12, 2024

Ok, I've tried the libvirt tests locally to see what the difference is and they fail in the same way. @liudalibj - did you have them working in #1935? is there some catch I'm doing wrong?

Update I decided to be sensible and check the nightly libvrt tests and they are working, so the failure is probably either user error, or related to us not using the latest CAA by default?

Copy link
Member

@wainersm wainersm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm it fix the compilation issue. Thanks @stevenhorsman !

Copy link
Member

@liudalibj liudalibj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevenhorsman I didn't look into this issue yet. thanks the fix!
LGTM

@@ -101,10 +101,11 @@ func TestDockerKbsKeyRelease(t *testing.T) {
if !isTestWithKbs() {
t.Skip("Skipping kbs related test as kbs is not deployed")
}
_ = keyBrokerService.EnableKbsCustomizedPolicy("deny_all.rego")
keyBrokerService.SetSampleSecretKey()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't have the return _ val, maybe won't pass lint check.

@stevenhorsman stevenhorsman marked this pull request as ready for review August 13, 2024 08:18
As part of the work in confidential-containers#1935, EnableKbsCustomizedPolicy was
replaced with EnableKbsCustomizedResourcePolicy and
EnableKbsCustomizedAttestationPolicy, but the docker tests
wasn't updated, so it doesn't compile, so fix this

Signed-off-by: stevenhorsman <[email protected]>
@stevenhorsman stevenhorsman force-pushed the docker-provider-kbs-fix branch from 223bc8e to 448ee25 Compare August 13, 2024 08:20
@stevenhorsman
Copy link
Member Author

I think we should merge this as it unblocks multiple other PRs, but I'm not super happy that I couldn't get this working properly locally, so we might have to fix something here as part of the docker e2e tests

@stevenhorsman stevenhorsman merged commit f1158cb into confidential-containers:main Aug 13, 2024
20 checks passed
@stevenhorsman stevenhorsman deleted the docker-provider-kbs-fix branch August 13, 2024 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants