Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large test matrix should be runnable on a schedule. #105

Closed
mark-rushakoff opened this issue May 24, 2022 · 2 comments
Closed

Large test matrix should be runnable on a schedule. #105

mark-rushakoff opened this issue May 24, 2022 · 2 comments

Comments

@mark-rushakoff
Copy link
Member

Related to #27, we should run ibctest against that large matrix on a schedule. We don't want the large matrix to gate ibctest CI, because some tests will likely fail on a large enough set of chains.

This might run on GitHub Actions or it might run somewhere else. We should also consider where this will archive the test report output.

@agouin
Copy link
Member

agouin commented May 24, 2022

One thought here, we don't need to re-run for existing tags, assuming chains don't delete and re-create tags. It would be nice if the suite could use the latest tag in the matrix file for the chain sets, so we are always testing the latest version of the chain on the scheduled runs. But then we could build caching that would only run the conformance tests for a chain set if the checksum changed for the latest tag of the docker image for either of the involved chains.

@jonathanpberger jonathanpberger changed the title Run large test matrix on a schedule Large test matrix should be runnable on a schedule. Mar 20, 2023
@Reecepbcups
Copy link
Member

no longer applies with the removal of redundent testing

@Reecepbcups Reecepbcups closed this as not planned Won't fix, can't repro, duplicate, stale Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants