Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes to multiple failing E2E tests #10057

Open
wants to merge 15 commits into
base: develop
Choose a base branch
from
Open

Conversation

alopezari
Copy link
Contributor

@alopezari alopezari commented Jan 3, 2025

Fixes #9746
Fixes #10115

Changes proposed in this Pull Request

This PR fixes the following failing E2E tests:

  • tests/e2e/specs/wcpay/merchant/merchant-payment-gateways-confirmation.spec.js:
    These were failing because of wrong locator that seemed to have been outdated at some point.

  • tests/e2e/specs/wcpay/shopper/shopper-checkout-failures.spec.js:
    Same as before, the tests were failing due to inaccurate locators.

  • tests/e2e/specs/wcpay/shopper/shopper-pay-for-order.spec.js:
    Inaccurate locators again.

  • Shopper tests in WooCommerce Beta version:
    WooCommerce 9.6 enables by default the Coming Soon mode for new sites, which breaks our tests. In order to bypass this, the script that sets up our test environment will deactivate this mode by disabling a flag in the Options table.

  • tests/e2e/specs/subscriptions/merchant/merchant-subscriptions-renew-action-scheduler.spec.js:
    WooCommerce 9.6 adds a new logic to define which fields are shown by default in the checkout form, and the company field from the billing section will now be hidden by default, unless you enable it. We've updated our test environment setup flow to enable the company field - used by our tests - in the checkout form by enabling a flag in the Options table.

  • tests/e2e/specs/wcpay/shopper/shopper-multi-currency-widget.spec.js:
    There seems to be a race condition here that make a test from this spec fail on CI but pass on my local environment, so I can't debug it properly. After spending some time with this one, I've decided to skip this test because it will be migrated soon and we don't need to spend some more extra time on it.

Testing instructions

  • Check that all checks passed for this PR.
  • Note that the E2E Tests - All workflow passed for this branch.
  • Set up the E2E env with by following the steps described here.
  • Run the merchant-payment-gateways-confirmation tests in headless mode with npm run test:e2e tests/e2e/specs/wcpay/merchant/merchant-payment-gateways-confirmation.spec.js.
  • Run the merchant-payment-gateways-confirmation tests in UI mode with npm run test:e2e-dev tests/e2e/specs/wcpay/merchant/merchant-payment-gateways-confirmation.spec.js.
  • Run the shopper-checkout-failures tests in headless mode with npm run test:e2e tests/e2e/specs/wcpay/shopper/shopper-checkout-failures.spec.js.
  • Run the shopper-checkout-failures tests in UI mode with npm run test:e2e-dev tests/e2e/specs/wcpay/shopper/shopper-checkout-failures.spec.js.
  • Run the shopper-pay-for-order tests in headless mode with npm run test:e2e tests/e2e/specs/wcpay/shopper/shopper-pay-for-order.spec.js.
  • Run the shopper-pay-for-order tests in UI mode with npm run test:e2e-dev tests/e2e/specs/wcpay/shopper/shopper-pay-for-order.spec.js.
  • Run the shopper-multi-currency-widget tests in headless mode with npm run test:e2e tests/e2e/specs/wcpay/shopper/shopper-multi-currency-widget.spec.js.
  • Run the shopper-multi-currency-widget tests in UI mode with npm run test:e2e-dev tests/e2e/specs/wcpay/shopper/shopper-multi-currency-widget.spec.js.
  • Reset the environment with npm run test:e2e-reset.
  • Add E2E_WC_VERSION=beta to your local.env file, this will configure your environment to use the latest WooCommerce Beta version.
  • Set up the environment again with npm run test:e2e-setup.
  • Run the merchant-orders-status-change tests in headless mode with npm run test:e2e tests/e2e/specs/wcpay/merchant/merchant-orders-status-change.spec.js.
  • Run the merchant-orders-status-change tests in UI mode with npm run test:e2e-dev tests/e2e/specs/wcpay/merchant/merchant-orders-status-change.spec.js.
  • Run the merchant-subscriptions-renew-action-scheduler tests in headless mode with npm run test:e2e tests/e2e/specs/subscriptions/merchant/merchant-subscriptions-renew-action-scheduler.spec.js.
  • Run the merchant-subscriptions-renew-action-scheduler tests in UI mode with npm run test:e2e-dev tests/e2e/specs/subscriptions/merchant/merchant-subscriptions-renew-action-scheduler.spec.js.
  • Finally, restore your test environment to use the latest standard WooCommerce version with npm run test:e2e-reset and removing E2E_WC_VERSION=beta from your local.env file.

  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@alopezari alopezari self-assigned this Jan 3, 2025
@botwoo
Copy link
Collaborator

botwoo commented Jan 3, 2025

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 10057 or branch name fix/failing-e2e-tests in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: 45bc1ed
  • Build time: 2025-01-10 14:30:06 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Jan 3, 2025

Size Change: 0 B

Total Size: 1.36 MB

ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.37 kB
release/woocommerce-payments/assets/css/admin.rtl.css 1.37 kB
release/woocommerce-payments/assets/css/success.css 182 B
release/woocommerce-payments/assets/css/success.rtl.css 184 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 2.63 kB
release/woocommerce-payments/dist/blocks-checkout.css 2.63 kB
release/woocommerce-payments/dist/blocks-checkout.js 55.6 kB
release/woocommerce-payments/dist/cart-block.js 17.2 kB
release/woocommerce-payments/dist/cart.js 5.73 kB
release/woocommerce-payments/dist/checkout-rtl.css 1.13 kB
release/woocommerce-payments/dist/checkout.css 1.13 kB
release/woocommerce-payments/dist/checkout.js 33.6 kB
release/woocommerce-payments/dist/express-checkout-rtl.css 229 B
release/woocommerce-payments/dist/express-checkout.css 229 B
release/woocommerce-payments/dist/express-checkout.js 15.7 kB
release/woocommerce-payments/dist/frontend-tracks.js 854 B
release/woocommerce-payments/dist/index-rtl.css 39.5 kB
release/woocommerce-payments/dist/index.css 39.4 kB
release/woocommerce-payments/dist/index.js 303 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.08 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 4.47 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 60.9 kB
release/woocommerce-payments/dist/multi-currency.css 4.47 kB
release/woocommerce-payments/dist/multi-currency.js 57.6 kB
release/woocommerce-payments/dist/order-rtl.css 730 B
release/woocommerce-payments/dist/order.css 730 B
release/woocommerce-payments/dist/order.js 42.4 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.33 kB
release/woocommerce-payments/dist/payment-gateways.css 1.33 kB
release/woocommerce-payments/dist/payment-gateways.js 38.8 kB
release/woocommerce-payments/dist/plugins-page-rtl.css 386 B
release/woocommerce-payments/dist/plugins-page.css 386 B
release/woocommerce-payments/dist/plugins-page.js 20.1 kB
release/woocommerce-payments/dist/product-details-rtl.css 433 B
release/woocommerce-payments/dist/product-details.css 436 B
release/woocommerce-payments/dist/product-details.js 12.5 kB
release/woocommerce-payments/dist/settings-rtl.css 11.7 kB
release/woocommerce-payments/dist/settings.css 11.6 kB
release/woocommerce-payments/dist/settings.js 225 kB
release/woocommerce-payments/dist/subscription-edit-page.js 703 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 524 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 20.2 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 730 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 19.3 kB
release/woocommerce-payments/dist/tokenized-express-checkout-rtl.css 229 B
release/woocommerce-payments/dist/tokenized-express-checkout.css 229 B
release/woocommerce-payments/dist/tokenized-express-checkout.js 16.6 kB
release/woocommerce-payments/dist/tos-rtl.css 235 B
release/woocommerce-payments/dist/tos.css 235 B
release/woocommerce-payments/dist/tos.js 21.8 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 6.13 kB
release/woocommerce-payments/dist/woopay-express-button.js 25 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.31 kB
release/woocommerce-payments/dist/woopay.css 4.28 kB
release/woocommerce-payments/dist/woopay.js 71 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 625 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 814 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.46 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/jetpack-script-data.js 772 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.02 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/script-data.js 69 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/babel.config.js 163 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.js 14.2 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.rtl.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.js 28.4 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.rtl.css 10 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.js 280 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.rtl.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.css 625 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.js 333 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.rtl.css 626 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-users.js 424 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 585 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.css 215 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.css 721 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.js 412 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-users.js 632 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.04 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 294 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 408 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.59 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 301 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 746 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 574 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 414 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 543 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.78 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.84 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 545 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.2 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.7 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 507 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 358 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 428 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 782 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.09 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.26 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 391 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.04 kB

compressed-size-action

@alopezari alopezari marked this pull request as ready for review January 3, 2025 16:25
@alopezari alopezari marked this pull request as draft January 3, 2025 17:00
@alopezari alopezari marked this pull request as ready for review January 11, 2025 09:30
@alopezari alopezari changed the title WIP: Fixes to multiple failing E2E tests Fixes to multiple failing E2E tests Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disable "coming soon" mode on E2E tests Persistent failing E2E tests
2 participants