Dev: Fix flaky and broken E2E tests #9183
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Test the buildOption 1. Jetpack Beta
Option 2. Jurassic Ninja - available for logged-in A12s🚀 Launch a JN site with this branch 🚀 ℹ️ Install this Tampermonkey script to get more options. Build info:
Note: the build is updated when a new commit is pushed to this PR. |
|
Size Change: 0 B Total Size: 1.33 MB ℹ️ View Unchanged
|
…ckable by puppeteer
danielmx-dev
requested review from
a team and
mdmoore
and removed request for
a team
danielmx-dev
changed the title
danielmx-dev
changed the title
… confirmation test
…e clicking the button
mdmoore
approved these changes
Jul 31, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to #9132
Changes proposed in this Pull Request
In #9132, we performed several fixes to make the E2E tests more stable.
After some runs, we found some issues with the following tests:
FLAKY Merchant Refund Failures in WC 7.7.0: I wasn't able to reproduce this locally, but it seems that in CI the Refund button is sometimes hidden behind the page header. Since this can't be reproduced locally we can just hide the header for this particular test as a workaround. If the issue keeps happening we may need to create a separate issue to track the root cause and fix it.
BROKEN Merchant progressive onboarding in all versions: It seems that for some reason the order of some elements changed. Selecting by
textcontent should fix the issue.BROKEN Shopper Klarna test: It looks like Klarna performs a lot of updates in their playground. This is a last attempt to fix the test while also providing a bit more flexibility regarding the purchase flow; however, we need to reconsider what's the best way to test the integration without depending on Klarna's UI since it's completely out of our control (and changes a lot!).
FLAKY Playwright Shopper 3DS test: It looks like it's having issues waiting for the iframe to appear, preventing the test from accepting the challenge.
BROKEN Merchant payment gateways confirmation: Newer WC versions will get rid of the "Save changes" button and will save immediately any action in the payment settings page.
Testing instructions
tests/e2e/README.mdto run the tests locally.npm run changelogto add a changelog file, choosepatchto leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.Post merge