Add control plane integration tests #223
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Our Python client doesn't have true integration tests. Currently, we have tests under
/integ/test_index.py
which mock all the requests and responses and primarily stress the connections between various elements of the client codebase. Fleshing out integration tests would help us have more confidence when making changes to the Python client, especially when involving the network layer.Solution
The plan is to split between control plane and data plane tests. For PRs I thought it'd be reasonable to split things out, and then finish off with CI changes to bring everything together.
/tests/integration/control/
test_create_index.py
test_configure_index.py
test_describe_index.py
test_list_indexes.py
Currently, the entire control plane suite completes in about ~35 seconds or so.
sdk-python-testing
currently has a Pod Limit of 7, and these seem to complete fine, but I'd like to get quote bumped to reduce possibility of flaking.**Question: ** Should I look at rolling some collection tests here as well?
Type of Change
Test Plan
Currently, these tests aren't being run in CI. That will come in a following PR. The environment variables that are used will be setup in GitHub secrets for the repo/workflows.
To test locally, you'll need to set environment variables and then call
pytest
: