Setup integration tests #619

fitzthum · 2024-12-10T20:23:07Z

Creates a framework for writing Trustee integration tests in Rust.

Put your tests in integration-tests/tests/integration.rs (or a similar file) and they will run when you do cargo test at the top level.

I've added two simple integration tests. They aren't anything beyond what we already have in other tests, but they add a structure that should help us to create a lot more. Before I get too carried away, please take a look at the setup.

Note that running the KBS in the background was trickier than expected. I think I found a pretty good way to do it in the end.

Without this we can't create a token configuration from outside the crate. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

mkulke · 2024-12-11T07:57:26Z

kbs/src/api_server.rs

@@ -129,6 +129,51 @@ impl ApiServer {
            .await
            .map_err(|e| Error::HTTPFailed { source: e.into() })
    }
+
+    /// Setup API server
+    pub fn server(self) -> Result<actix_web::dev::Server> {


this seems to share a lot of logic with serve() in the same file, can we share the code?

Yeah I don't like this duplication. I tried a bunch of things to reduce it but I could not get any of them to work. The actix crate is a bit fiddly. For instance I tried having just one function with a flag, but that doesn't work since the function needs to be async in one case but not in the other. Then I tried making serve a wrapper of server. I thought I could just return the actix HttpServer from server and then run it either in the test or in serve. This didn't work for reasons that are sort of hard to describe quickly. Anyway I ran out of ideas here (I'll skip over my other, worse, attempts), but if anyone else has any I would like to avoid duplicating this. Maybe we could get rid of one if we reworked the kbs.rs binary main.

Ok, I tried a bunch more things and I finally have something that seems to work and avoids the duplication. PTAL

Xynnn007 · 2024-12-11T08:19:01Z

Looks great! One question before getting into the code:

why not we directly have an tests directory under kbs to have these integration tests, like what we do in image-rs

A quick glance is that way we need to add kbs_client to kbs's [dev-dependencies] section in Cargo.toml.

This way, we could avoid another sub crate.

fitzthum · 2024-12-11T15:43:39Z

why not we directly have an tests directory under kbs to have these integration tests, like what we do in image-rs

I think this would work. I see the integration tests as having a broader scope than just the KBS, so I made a new package, but if that adds overhead I could move it. Generally I think that since the repo merge, we've had some stuff in individual packages that should really be in the top of the repo (like configs and docs, perhaps).

Xynnn007

I think it would make sense to have a separate crate if we want to add more tests. One question is that we are having some e2e test

docker-compose ci
e2e test here
this PR

Do we need to delete some duplicated ones?

integration-tests/tests/integration.rs

integration-tests/Cargo.toml

Xynnn007 · 2024-12-12T01:36:03Z

integration-tests/tests/integration.rs

+    allow = false
+";
+
+#[allow(dead_code)]


Why do we need this lint macro?

The Custom and DenyAll variants are never instantiated. I could move the dead code thing just to those variants but then I'd have to use it twice.

integration-tests/tests/integration.rs

fitzthum · 2024-12-12T20:01:38Z

@Xynnn007

Do we need to delete some duplicated ones?

Since these integration tests use the crate directly, they don't test the binaries or the docker-compose configuration or the k8s configuration. I think it's fine to keep some simple tests around that make sure that those interfaces still work. We should do a little bit of cleanup and organization with our existing tests, but I think that's for another PR.

The existing serve function returns a future that must be awaited on for the server to run. This causes the caller to block and does not allow the server to be halted unless the process is terminated. This is not ideal when using the server as a crate (or in a test). Instead, add a function that returns a server which can then be awaited on or run in the background. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

To support integration tests written in rust, let's add a new package (which will only be used internally) called integration-tests. Inside the package, we have a tests repo where we can put tests. The package imports other trustee packages using a local path, so it should always have the most recent version of things. Now integration tests will run simply by doing cargo test at the top of the repo. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

Let the make check target (which maybe should be renamed?) also run the integration tests. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

The OPA linter seems to have updated and now complains if we don't use an if before our blocks. This is unrelated to the rest of this PR. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

token: make token module public

050b619

Without this we can't create a token configuration from outside the crate. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

fitzthum requested a review from a team as a code owner December 10, 2024 20:23

mkulke reviewed Dec 11, 2024

View reviewed changes

fitzthum force-pushed the setup-integration-tests branch from 1adf56a to b311ab0 Compare December 11, 2024 19:15

Xynnn007 reviewed Dec 12, 2024

View reviewed changes

fitzthum force-pushed the setup-integration-tests branch from b311ab0 to 5d3767a Compare December 12, 2024 19:55

fitzthum force-pushed the setup-integration-tests branch 3 times, most recently from 669f8e6 to 75dee21 Compare December 20, 2024 22:45

fitzthum added 3 commits December 20, 2024 16:58

tests: add integration test to Makefile

8ffcff3

Let the make check target (which maybe should be renamed?) also run the integration tests. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

fitzthum force-pushed the setup-integration-tests branch 2 times, most recently from 89754d0 to fb4069f Compare December 20, 2024 23:12

policy: fixup simple default policy

ce92336

The OPA linter seems to have updated and now complains if we don't use an if before our blocks. This is unrelated to the rest of this PR. Signed-off-by: Tobin Feldman-Fitzthum <[email protected]>

fitzthum force-pushed the setup-integration-tests branch from fb4069f to ce92336 Compare December 20, 2024 23:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup integration tests #619

Setup integration tests #619

fitzthum commented Dec 10, 2024

mkulke Dec 11, 2024

fitzthum Dec 11, 2024 •

edited

Loading

fitzthum Dec 20, 2024 •

edited

Loading

Xynnn007 commented Dec 11, 2024

fitzthum commented Dec 11, 2024 •

edited

Loading

Xynnn007 left a comment

Xynnn007 Dec 12, 2024

fitzthum Dec 12, 2024 •

edited

Loading

fitzthum commented Dec 12, 2024

Setup integration tests #619

Are you sure you want to change the base?

Setup integration tests #619

Conversation

fitzthum commented Dec 10, 2024

mkulke Dec 11, 2024

Choose a reason for hiding this comment

fitzthum Dec 11, 2024 • edited Loading

Choose a reason for hiding this comment

fitzthum Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Xynnn007 commented Dec 11, 2024

fitzthum commented Dec 11, 2024 • edited Loading

Xynnn007 left a comment

Choose a reason for hiding this comment

Xynnn007 Dec 12, 2024

Choose a reason for hiding this comment

fitzthum Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

fitzthum commented Dec 12, 2024

fitzthum Dec 11, 2024 •

edited

Loading

fitzthum Dec 20, 2024 •

edited

Loading

fitzthum commented Dec 11, 2024 •

edited

Loading

fitzthum Dec 12, 2024 •

edited

Loading