Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a job for running the HTML API tests. #7602

Closed
wants to merge 23 commits into from
+26 −4
Closed

Add a job for running the HTML API tests. #7602

Changes from 5 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
55b9106
Add a job for running the HTML API tests.
desrosj Oct 21, 2024
ced4c98
Pass group to reusable workflow.
desrosj Oct 21, 2024
4d8b375
Use correct input name.
desrosj Oct 21, 2024
405a919
Create new matrix for specific groups.
desrosj Oct 21, 2024
48708a8
Merge remote-tracking branch 'upstream/trunk' into try/html5lib-tests…
desrosj Nov 7, 2024
5ebc940
Merge remote-tracking branch 'upstream/trunk' into try/html5lib-tests…
desrosj Dec 17, 2024
693e642
Try pulling specific test groups up to top.
desrosj Dec 17, 2024
8bc2478
Use the latest LTS version of MySQL.
desrosj Dec 17, 2024
91dbe73
Fix matrix value reference.
desrosj Dec 17, 2024
3b88447
Try different nesting.
desrosj Dec 17, 2024
6f9c9f1
Tweak nesting again.
desrosj Dec 17, 2024
40aade0
Use workflow in PR branch.
desrosj Dec 17, 2024
f2d849d
Does nesting only work for one `/`?
desrosj Dec 17, 2024
c196823
Correctly nest with or without test group input.
desrosj Dec 17, 2024
094471a
Clean up conditions.
desrosj Dec 17, 2024
f2110f0
Final name cleanup.
desrosj Dec 17, 2024
52b2d32
Remove log file unintentionally included.
desrosj Dec 17, 2024
db91376
Confirm multiple combinations build out names correctly.
desrosj Dec 17, 2024
f0b6f9a
Reset the test matrix to remove unnecessary combos.
desrosj Dec 17, 2024
e1925e1
Rely on default values for inputs when possible.
desrosj Dec 17, 2024
76ff954
Include DB version to avoid failure from default changes.
desrosj Dec 17, 2024
ea9e3c2
Add a few more PHP versions.
desrosj Dec 17, 2024
1388d88
Merge remote-tracking branch 'upstream/trunk' into try/html5lib-tests…
desrosj Dec 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion .github/workflows/phpunit-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,13 +154,45 @@ jobs:
phpunit-config: ${{ matrix.multisite && 'tests/phpunit/multisite.xml' || 'phpunit.xml.dist' }}
report: ${{ matrix.report || false }}

#
# Runs specific test groups.
#
specific-test-groups:
name: PHP ${{ matrix.php }}
desrosj marked this conversation as resolved.
Show resolved Hide resolved
uses: WordPress/wordpress-develop/.github/workflows/reusable-phpunit-tests-v3.yml@trunk
permissions:
contents: read
secrets: inherit
if: ${{ github.repository == 'WordPress/wordpress-develop' || github.event_name == 'pull_request' }}
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest ]
php: [ '8.3' ]
db-type: [ 'mysql' ]
db-version: [ '8.0' ]
multisite: [ false ]
memcached: [ false ]
phpunit-test-groups: [ 'html-api-html5lib-tests' ]

with:
os: ${{ matrix.os }}
php: ${{ matrix.php }}
db-type: ${{ matrix.db-type }}
db-version: ${{ matrix.db-version }}
multisite: ${{ matrix.multisite }}
memcached: ${{ matrix.memcached }}
phpunit-config: ${{ matrix.multisite && 'tests/phpunit/multisite.xml' || 'phpunit.xml.dist' }}
phpunit-test-groups: ${{ matrix.phpunit-test-groups }}
report: ${{ matrix.report || false }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not report? I believe this flag is for this:

Submit test results to the WordPress.org host test results

I don't know what that implies or means exactly, but these tests should really have the same weight as any other phpunit tests. They should not fail on anything merged for release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tells the workflow to submit the results of running the test suite to the w.org hosting tests.

Currently, the hosting tests does support submitting multiple reports per commit, but it does not support submitting subsets of tests. The assumption is that you run the full test suite, there would be no indication that a report only represents a small segment.

I think, if we wanted to include the results of these tests when submitting, the reusable workflow would need to be updated to somehow include the provided test group and not only group excluding the standard phpunit.xml setup. This could still run on just one job, but it would then again become nested under the PHP version where it was included.

Let's start without including it in a report and we can always look to include it in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Sure, that sounds good. Just getting these tests running on CIis a big benefit.


slack-notifications:
name: Slack Notifications
uses: WordPress/wordpress-develop/.github/workflows/slack-notifications.yml@trunk
permissions:
actions: read
contents: read
needs: [ test-with-mysql, test-with-mariadb ]
needs: [ test-with-mysql, test-with-mariadb, specific-test-groups ]
if: ${{ github.repository == 'WordPress/wordpress-develop' && github.event_name != 'pull_request' && always() }}
with:
calling_status: ${{ contains( needs.*.result, 'cancelled' ) && 'cancelled' || contains( needs.*.result, 'failure' ) && 'failure' || 'success' }}
Expand Down
Loading