YML-based test automation

[christophwille]

Triggered by #3 (comment)

The inner-loop tests should definitely remain in the current unit test project. However, supporting running build verification tests using eg https://github.com/open-policy-agent/opa/tree/main/test/cases/testdata is definitely desireable.

Maybe wrap this in a separate test project so they are split from the inner loop tests. How to include the tests is the 1m question... copy is a no-no, on the other hand I don't want to drag in submodules "just" for this. Ideas welcome.

[christophwille]

To pick on a specific example: iinuwa@a73dbc7

The C#-implemented SDK builtins should be tested directly from a C# unit test, not indirectly via rego (I would consider the latter an "integration test").

[christophwille]

open-policy-agent/opa#3083 (comment)

[iinuwa]

How to include the tests is the 1m question... copy is a no-no, on the other hand I don't want to drag in submodules "just" for this. Ideas welcome.

What do you mean by copying here? Do you mean you don't want files checked into the repo? If so, the npm-opa-sdk CI downloads the opa repo as a tar and then extracts the subdirectory while running the tests.

The C#-implemented SDK builtins should be tested directly from a C# unit test, not indirectly via rego (I would consider the latter an "integration test").

Agreed. Are you saying that you want the YAML-based tests to be unit or integration tests?

[christophwille]

Correct, I don't want the files in this repo - the build automation doing the downloading could be an option. And when I am talking about those "third party tests", I consider them "SDK validation tests" (in a sense we are reusing somebody else's unit tests)