Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E Tests: Only install Chromium and its deps for Playwright #40827

Merged
merged 2 commits into from
Jan 3, 2025

Conversation

tbradsha
Copy link
Contributor

@tbradsha tbradsha commented Jan 2, 2025

Right now we use default settings to install Chromium, Firefox, and Webkit for Playwright prior to running E2E tests, but we only use Chromium in our config. GH Actions also shows missing dependencies. This PR fixes both of those by switching from this:

playwright install

to this:

playwright install --with-deps chromium

It trades about 10 seconds of unneeded installs for about 15 seconds of dependencies, which means this is technically slower by a few seconds. We could remove the --with-deps if we prefer since tests run fine anyway, though I don't see a way to suppress those dependency warnings.

I considered --only-shell, but this would force headless on local installs, which isn't ideal.

Other information:

  • Have you written new tests for your changes, if applicable?
  • Have you checked the E2E test CI results, and verified that your changes do not break them?
  • Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

  • Is CI happy?
  • Is only Chromium (and FFMPEG) downloading)?
  • Is the dependency warning gone?
  • In particular, are E2E tests passing (as well as they did before)?

Here's an old run with multiple browsers downloaded and the dependency warning:
https://github.com/Automattic/jetpack/actions/runs/12585666388/job/35077965663

Copy link
Contributor

github-actions bot commented Jan 2, 2025

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

  • To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the fix/playwright_install_tweaks branch.

  • To test on Simple, run the following command on your sandbox:

    bin/jetpack-downloader test jetpack fix/playwright_install_tweaks
    

Interested in more tips and information?

  • In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
  • Read more about our development workflow here: PCYsg-eg0-p2
  • Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

@github-actions github-actions bot added [Plugin] Automattic For Agencies Client [Plugin] Boost A feature to speed up the site and improve performance. [Plugin] Classic Theme Helper Plugin [Plugin] Jetpack Issues about the Jetpack plugin. https://wordpress.org/plugins/jetpack/ [Plugin] Search A plugin to add an instant search modal to your site to help visitors find content faster. [Plugin] Social Issues about the Jetpack Social plugin [Plugin] Starter Plugin [Plugin] VideoPress A standalone plugin to add high-quality VideoPress videos to your site. [Tests] Includes Tests labels Jan 2, 2025
Copy link
Contributor

github-actions bot commented Jan 2, 2025

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

  • ✅ Include a description of your PR changes.
  • ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  • ✅ Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
  • ✅ Add testing instructions.
  • ✅ Specify whether this PR includes any changes to data or privacy.
  • ✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖


The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.


Follow this PR Review Process:

  1. Ensure all required checks appearing at the bottom of this PR are passing.
  2. Choose a review path based on your changes:
    • A. Team Review: add the "[Status] Needs Team Review" label
      • For most changes, including minor cross-team impacts.
      • Example: Updating a team-specific component or a small change to a shared library.
    • B. Crew Review: add the "[Status] Needs Review" label
      • For significant changes to core functionality.
      • Example: Major updates to a shared library or complex features.
    • C. Both: Start with Team, then request Crew
      • For complex changes or when you need extra confidence.
      • Example: Refactor affecting multiple systems.
  3. Get at least one approval before merging.

Still unsure? Reach out in #jetpack-developers for guidance!

Copy link
Contributor

@robertsreberski robertsreberski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a good optimization for me, thanks for implementing that!

It might be interesting to include the context of how did you figure out the issue.

@tbradsha
Copy link
Contributor Author

tbradsha commented Jan 3, 2025

It might be interesting to include the context of how did you figure out the issue.

In this case I saw the output in the GH Action logs downloading multiple browsers and showing the dependency error, knew we only used one browser, and so updated our process to download the one with its dependencies.

@tbradsha tbradsha merged commit 7d3e848 into trunk Jan 3, 2025
74 of 77 checks passed
@tbradsha tbradsha deleted the fix/playwright_install_tweaks branch January 3, 2025 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Tests [Plugin] Automattic For Agencies Client [Plugin] Boost A feature to speed up the site and improve performance. [Plugin] Classic Theme Helper Plugin [Plugin] Jetpack Issues about the Jetpack plugin. https://wordpress.org/plugins/jetpack/ [Plugin] Search A plugin to add an instant search modal to your site to help visitors find content faster. [Plugin] Social Issues about the Jetpack Social plugin [Plugin] Starter Plugin [Plugin] VideoPress A standalone plugin to add high-quality VideoPress videos to your site. [Tests] Includes Tests [Type] Janitorial
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants