Add a job for running the HTML API tests.

[desrosj]

Alternative approach to #6963.

Trac ticket: https://core.trac.wordpress.org/ticket/61209

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props desrosj, jonsurrell, jorbin.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[desrosj]

@sirreal @aaronjorbin What do you think about this approach?

[sirreal]

This is great. I really want to get these tests running, thank you.

I've left notes and questions but nothing blocking.

[sirreal]

Why not report? I believe this flag is for this:

Submit test results to the WordPress.org host test results

I don't know what that implies or means exactly, but these tests should really have the same weight as any other phpunit tests. They should not fail on anything merged for release.

[desrosj]

This tells the workflow to submit the results of running the test suite to the w.org hosting tests.

Currently, the hosting tests does support submitting multiple reports per commit, but it does not support submitting subsets of tests. The assumption is that you run the full test suite, there would be no indication that a report only represents a small segment.

I think, if we wanted to include the results of these tests when submitting, the reusable workflow would need to be updated to somehow include the provided test group and not only group excluding the standard phpunit.xml setup. This could still run on just one job, but it would then again become nested under the PHP version where it was included.

Let's start without including it in a report and we can always look to include it in the future.

[sirreal]

👍 Sure, that sounds good. Just getting these tests running on CIis a big benefit.

[sirreal]

Is there anything I can do to help this land?

I think #6963 can be closed with this work superseding it.

[desrosj]

@sirreal I think this is good to go! Do you mind giving it a final look?

[sirreal]

This is great, thank you. The hierarchy of the tests here makes much more sense to me where the specific groups are top-level.

[desrosj]

Thanks all! Merged in https://core.trac.wordpress.org/changeset/59528.