Make Install Testing Workflow More Flexible

[desrosj]

This makes the install testing workflow a bit more flexible in order to allow usage for any branch of WordPress currently receiving security updates.

Because you can't manually start workflows for a PR (that I'm aware of), I've run the workflow manually within my fork. Here are a few test workflows:

• WordPress 4.1.20
• WordPress 5.0.18
• WordPress 5.2.1
• WordPress 6.0.6

Remaining items:

• The correct version of WP CLI is not installing on PHP 5.3, but is for 5.4-5.5.
• There are issues with the MySQL 5.5 container not starting correctly.
• The MySQL versions needs to be verified and confirmed.
• It's unlikely because of how WP versions and ships security updates, but Is there a more standard way to list version support?

Trac ticket: https://core.trac.wordpress.org/ticket/58977

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[desrosj]

Some updates here:

• I've added an exclusion for testing PHP 5.3 for now. While it would be nice to have this testing running, I don't think it should block this improvement.
• I'm still debugging why the MySQL 5.5 container won't start. I've also added this to the exclude list. We don't currently run tests against MySQL 5.5 and while it shouldn't be ignored (currently accounts for 15% of all sites), I also don't think this is a blocker.
• I've gone back and reviewed the MySQL version support. Historically, the changes to support have not been as well announced as PHP. New versions of MySQL usually just work with a few small compatibility issues that don't creep up immediately. Another review is appreciated, though.

[desrosj]

Here are a few test workflow runs within my fork to confirm that manually triggered runs work:

• WP 4.2.16
• WP 5.1.1
• WP 5.8.3
• Latest

[joemcgill]

This looks really useful. A couple of questions:

1. Why is it helpful to add a scheduled run on every Monday?
2. Is there an official documented page somewhere on WP.org that lists all the supported PHP/MySQL versions for each WP version that this can be cross-referenced against?

[desrosj]

1. Why is it helpful to add a scheduled run on every Monday?

My thinking is that regular testing of the nightly build against the complete matrix of supported versions will ensure no bugs in the install process are introduced. It's likely that folks are not regularly creating new installs using the nightly package very often.

2. Is there an official documented page somewhere on WP.org that lists all the supported PHP/MySQL versions for each WP version that this can be cross-referenced against?

There is for PHP versions, but not currently for MySQL. I think it's worth adding.

However, in my research, new MySQL versions do not have clear "supported" milestones. For PHP, it's always been "the first release following or immediately preceding a PHP version release". But for MySQL, it seemed to be "it should just work, but there may be some edge cases and bugs that are discovered along the way."

Using MySQL 5.7 as an example, it was released on October 21, 2015. WordPress 4.4 was the next major version released on November 25, 2015. WP 4.4 did bump the recommended version of MySQL from 5.5 to 5.6, but there are no tickets in the milestone specifically referencing MySQL 5.7. I looked through 4.9, but could not find any MySQL 5.7 compatibility issues (though Trac search is not great). I'll need to do some more historical research to try and find out if there's something I'm missing.

For sake of this PR, I used the MySQL version release dates and cross referenced them with the WP version release dates. I assumed that the first major version of WP after a MySQL release supported that version.

• WP 3.1 first release after MySQL 5.5
• WP 3.6 first release after MySQL 5.6
• WP 4.4 first release after MySQL 5.7
• WP 5.0 first release after MySQL 8.0

WP has also not historically trimmed support for MySQL versions. The current minimum required version sits at 5.0 😱 I plan to address this during the 6.5 cycle. Regardless, the oldest versions of MySQL available in Docker is 5.5.x, and that container fails to start in my testing.

[aaronjorbin]

I'm not sure that this or the version-support-php.json file should go in the root directory. A couple of alternative ideas:

• Make them hidden files. Since they are mostly designed to be machine readable rather than information that a human needs, this lowers their prominince
• Move them in to .github or maybe .github/data since the place they are meant to be consumed is github.

[desrosj]

I've gone and made the files invisible. I also added PHP 5.2 and MySQL 5.0/5.1 to the support files while excluding them in the workflows. This makes those files a more complete reference.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[aaronjorbin]

Thanks @desrosj!

[desrosj]

Merged in https://core.trac.wordpress.org/changeset/57218.