Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add Ecosystem tests #521

Merged
merged 20 commits into from
Feb 5, 2025
Merged

CI: Add Ecosystem tests #521

merged 20 commits into from
Feb 5, 2025

Conversation

bkchr
Copy link
Contributor

@bkchr bkchr commented Dec 26, 2024

This adds the ecosystem tests as a CI job.

  • Does not require a CHANGELOG entry

@bkchr
Copy link
Contributor Author

bkchr commented Dec 28, 2024

@xlc any idea why the ecosystem tests are failing: https://github.com/polkadot-fellows/runtimes/actions/runs/12518886514/job/34922591260?pr=521 ?

@xlc
Copy link
Contributor

xlc commented Jan 3, 2025

The error is Error handling request: 'RpcError: -32008: Response is too big: Exceeded max limit of 1048576 so it is requesting some big value from RPC and failed. There are a few similar intermittent errors in the nightly runs of the ecosystem tests and I haven't got the chance to investigate on exactly what state caused the error and why it is only an Intermittent error (rerun fixes the error most of the time).
Will spend some time on this. Meanwhile, I will suggest setup some retry logic. e.g. https://github.com/open-web3-stack/polkadot-ecosystem-tests/blob/f6b76c9201dfca0c1f95e14840ccfed2029efd02/.github/workflows/update-known-good.yml#L105

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

The error is Error handling request: 'RpcError: -32008: Response is too big: Exceeded max limit of 1048576 so it is requesting some big value from RPC and failed. There are a few similar intermittent errors in the nightly runs of the ecosystem tests and I haven't got the chance to investigate on exactly what state caused the error and why it is only an Intermittent error (rerun fixes the error most of the time).

Maybe we need a dedicated RPC server with a higher response limit.

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

What is weird is that it already fails on getting the runtime version:

getRuntimeVersion(at?: BlockHash): RuntimeVersion:: -32603: Internal RpcError: -32008: Response is too big: Exceeded max limit of 1048576

@xlc
Copy link
Contributor

xlc commented Jan 3, 2025

it is possible to set rpc endpoints via env vars and we can use github action secrets to avoid expose internal rpcs to public

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

What is also weird is that the default max response size is 15MIB. Meaning they have decreased the size quite a lot?

@bkchr bkchr mentioned this pull request Jan 7, 2025
6 tasks
@bkchr
Copy link
Contributor Author

bkchr commented Jan 29, 2025

/merge

@fellowship-merge-bot fellowship-merge-bot bot enabled auto-merge (squash) January 29, 2025 20:09
@fellowship-merge-bot
Copy link
Contributor

Enabled auto-merge in Pull Request

Available commands
  • /merge: Enables auto-merge for Pull Request
  • /merge cancel: Cancels auto-merge for Pull Request
  • /merge help: Shows this menu

For more information see the documentation

@bkchr bkchr disabled auto-merge February 5, 2025 09:32
@bkchr bkchr merged commit 1a8ac56 into main Feb 5, 2025
23 checks passed
@bkchr bkchr deleted the bkchr-ecosystem-tests branch February 5, 2025 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants