Improve E2E test reliability by disabling "Coming Soon" mode and handling dynamic checkout fields

[eduardoumpierre]

Fixes #10115

Changes proposed in this Pull Request

This PR aims to enhance the reliability of our end-to-end tests by disabling the "Coming soon" mode (when enabled) and checking if the checkout fields exist before attempting to fill them. These changes aim to provide a consistent and robust testing environment for all scenarios.

• Setup step to disable "Coming Soon" mode: Introduces a setup step to reset the site configuration and ensure the "Coming Soon" mode is disabled before tests are executed. This guarantees the site is accessible for testing.
• Dynamic handling of checkout fields: Updates test logic to check for the presence of fields on the checkout page before attempting to fill them, accommodating variations based on site configuration.

Testing instructions

Before checking out to this branch

• Add E2E_WC_VERSION=beta to your tests/e2e/config/local.env
• Rebuild your E2E env by running npm run test:e2e-reset followed by npm run test:e2e-setup
• Run npm run test:e2e-pw-ui shopper-checkout-purchase
• Confirm that the site is failing due to the coming soon mode

Checkout to this branch

• Run the test again
• The tests should be passing
  • The "using a 3DS card" test can be a bit flaky due to the fixed timeout on it but it seems to be passing consistently on the CI
• Apply the following diff to remove the main change in the checkout fields handling:

diff --git a/tests/e2e-pw/utils/shopper.ts b/tests/e2e-pw/utils/shopper.ts
index 5a80d2ec4..36bbf0657 100644
--- a/tests/e2e-pw/utils/shopper.ts
+++ b/tests/e2e-pw/utils/shopper.ts
@@ -34,9 +34,9 @@ export const fillBillingAddress = async (
 		const element = page.locator( field.selector );
 		const value = billingAddress[ field.key ];
 
-		if ( ! ( await element.isVisible() ) ) {
-			continue;
-		}
+		// if ( ! ( await element.isVisible() ) ) {
+		// 	continue;
+		// }
 
 		if ( field.type === 'text' ) {
 			await element.fill( value );

• Run the test again
• The using a basic card test should fail

---

• Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
• Covered with tests (or have a good reason not to test in description ☝️)
• Tested on mobile (or does not apply)

Post merge

• Link to testing instructions from release testing doc following these instructions : Add link here / 'QA Testing Not Applicable'
• Add or update critical flows and testing instructions for critical flows, if applicable.
• Add what's changed (description, screenshot, demo videos etc.) to the release announcement post, if applicable.

[botwoo]

Test the build

Option 1. Jetpack Beta

• Install and activate Jetpack Beta.
• Use this build by searching for PR number 10117 or branch name dev/10115-disable-coming-soon-mode-for-e2e in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.

---

Build info:

• Latest commit: 99c330d
• Build time: 2025-01-08 22:59:10 UTC

Note: the build is updated when a new commit is pushed to this PR.

[github-actions]

Size Change: 0 B

Total Size: 1.36 MB

ℹ️ View Unchanged

Filename	Size
release/woocommerce-payments/assets/css/admin.css	1.37 kB
release/woocommerce-payments/assets/css/admin.rtl.css	1.37 kB
release/woocommerce-payments/assets/css/success.css	182 B
release/woocommerce-payments/assets/css/success.rtl.css	184 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css	2.63 kB
release/woocommerce-payments/dist/blocks-checkout.css	2.63 kB
release/woocommerce-payments/dist/blocks-checkout.js	55.6 kB
release/woocommerce-payments/dist/cart-block.js	17.2 kB
release/woocommerce-payments/dist/cart.js	5.73 kB
release/woocommerce-payments/dist/checkout-rtl.css	1.13 kB
release/woocommerce-payments/dist/checkout.css	1.13 kB
release/woocommerce-payments/dist/checkout.js	33.6 kB
release/woocommerce-payments/dist/express-checkout-rtl.css	229 B
release/woocommerce-payments/dist/express-checkout.css	229 B
release/woocommerce-payments/dist/express-checkout.js	15.7 kB
release/woocommerce-payments/dist/frontend-tracks.js	854 B
release/woocommerce-payments/dist/index-rtl.css	39.5 kB
release/woocommerce-payments/dist/index.css	39.4 kB
release/woocommerce-payments/dist/index.js	303 kB
release/woocommerce-payments/dist/multi-currency-analytics.js	1.08 kB
release/woocommerce-payments/dist/multi-currency-rtl.css	4.47 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js	60.9 kB
release/woocommerce-payments/dist/multi-currency.css	4.47 kB
release/woocommerce-payments/dist/multi-currency.js	57.6 kB
release/woocommerce-payments/dist/order-rtl.css	730 B
release/woocommerce-payments/dist/order.css	730 B
release/woocommerce-payments/dist/order.js	42.4 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css	1.33 kB
release/woocommerce-payments/dist/payment-gateways.css	1.33 kB
release/woocommerce-payments/dist/payment-gateways.js	38.8 kB
release/woocommerce-payments/dist/plugins-page-rtl.css	386 B
release/woocommerce-payments/dist/plugins-page.css	386 B
release/woocommerce-payments/dist/plugins-page.js	20.1 kB
release/woocommerce-payments/dist/product-details-rtl.css	433 B
release/woocommerce-payments/dist/product-details.css	436 B
release/woocommerce-payments/dist/product-details.js	12.5 kB
release/woocommerce-payments/dist/settings-rtl.css	11.7 kB
release/woocommerce-payments/dist/settings.css	11.6 kB
release/woocommerce-payments/dist/settings.js	225 kB
release/woocommerce-payments/dist/subscription-edit-page.js	703 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css	524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css	524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js	20.2 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js	730 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css	120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css	120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js	19.3 kB
release/woocommerce-payments/dist/tokenized-express-checkout-rtl.css	229 B
release/woocommerce-payments/dist/tokenized-express-checkout.css	229 B
release/woocommerce-payments/dist/tokenized-express-checkout.js	16.6 kB
release/woocommerce-payments/dist/tos-rtl.css	235 B
release/woocommerce-payments/dist/tos.css	235 B
release/woocommerce-payments/dist/tos.js	21.8 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js	6.13 kB
release/woocommerce-payments/dist/woopay-express-button.js	25 kB
release/woocommerce-payments/dist/woopay-rtl.css	4.31 kB
release/woocommerce-payments/dist/woopay.css	4.28 kB
release/woocommerce-payments/dist/woopay.js	71 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css	625 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js	814 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js	2.46 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/jetpack-script-data.js	772 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js	1.02 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/script-data.js	69 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/babel.config.js	163 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.css	2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.js	14.2 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.rtl.css	2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.css	10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.js	28.4 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.rtl.css	10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.css	198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.js	280 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.rtl.css	198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.css	625 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.js	333 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.rtl.css	626 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-users.js	424 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js	521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js	585 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.css	215 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.js	521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.css	721 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.js	412 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-users.js	632 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css	1.04 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css	294 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css	408 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css	3.59 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css	301 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css	746 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css	574 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css	414 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js	543 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js	9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js	6.78 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js	3.84 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js	545 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js	2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js	22.2 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js	11.7 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js	1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js	507 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js	358 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js	428 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js	1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js	782 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js	1.09 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js	1.26 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css	391 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js	3.04 kB

_{compressed-size-action}

[eduardoumpierre]

I'm closing this issue in favor of #10057 as discussed on p1736411853098899/1736362677.623219-slack-C01B8KNUYSW.