Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: 2.4.4* mirror building fails #186

Closed
fballiano opened this issue Nov 13, 2024 · 8 comments · Fixed by #187
Closed

bug: 2.4.4* mirror building fails #186

fballiano opened this issue Nov 13, 2024 · 8 comments · Fixed by #187

Comments

@fballiano
Copy link
Contributor

All 2.4.4 versions fail with these errros

Run diff -rq -x "*composer*" -x "*autoload*" -x "*mage---*" -x ".git" /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos
Only in /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/bin: yaml-lint
Only in /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/deprecation-contracts: .gitignore
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/deprecation-contracts/README.md and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/deprecation-contracts/README.md differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/deprecation-contracts/function.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/deprecation-contracts/function.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/CHANGELOG.md and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/CHANGELOG.md differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Command/LintCommand.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Command/LintCommand.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Dumper.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Dumper.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Escaper.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Escaper.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Exception/ParseException.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Exception/ParseException.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Inline.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Inline.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/LICENSE and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/LICENSE differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Parser.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Parser.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/README.md and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/README.md differ
Only in /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml: Resources
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Unescaper.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Unescaper.php differ
Files /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/magento/vendor/symfony/yaml/Yaml.php and /home/runner/work/generate-mirror-repo-js/generate-mirror-repo-js/mageos/vendor/symfony/yaml/Yaml.php differ
@rhoerr
Copy link
Contributor

rhoerr commented Nov 13, 2024

Some past messages on 2.4.4 mirror integrity failures that may or may not still be applicable:

Vinai Kopp🎊🏅 — 6/11/24, 3:56 PM
For now I'll keep the old 2.4.4 releases that install the older version of aws/aws-sdk-php as they are.
The alternative would be to re-build and re-tag those releases in the mirror.

Vinai Kopp🎊🏅 — 10/10/24, 5:55 AM
Otherwise 2.4.4-p1 to 2.4.4-p11 are broken because of differences in some symfony packages, that is, symfony/deprecation-contracts and symfony/yaml.
Mage-OS installs 5.0.11 of symfony/yaml, while upstream installs 5.3.14.
Will check why next
Regarding symfony/deprecation-contracts, Mage-OS installs 3.5.0, while upstream installs 2.5.3
...
I'm thinking of updating the mirror with the new releases, even though the 2.4.4-p1 to 2.4.4-p11 releases don't yet use the same versions for symfony/yaml and symfony/deprecation-contracts
I just don't know how long it will take to figure out what the difference is, given the recursive dependency tree is taking so long.
The composer --no-plugins --tree why symfony/deprecation-contracts process is still running, almost 21 hours later.
It uses about 100GB of virtual memory, and about 1GB of physical memory. From the outside I can see it is reading and writing to memory a lot, so clearly something is happening.
I do wonder if it will actually get done.

@fballiano
Copy link
Contributor Author

If I install 2.4.4-p11 (both upstream and mageos mirror) I get the same version of symfony/yaml

fab@MBP14-Fab-2 /t/244mirror> cat composer.lock |grep yaml
                "ext-yaml": "*",
                "symfony/yaml": "^4.4|^5.0|^6.0"
                "symfony/yaml": "To use the yaml reference dumper"
                "symfony/yaml": "<4.4.26"
                "symfony/yaml": "^4.4.26|^5.0|^6.0"
                "symfony/yaml": ""
                    "generated/yaml.php",
                "symfony/yaml": "~3|~4|~5|~6|~7"
                "symfony/yaml": "If you want to parse features, represented in YAML files"
                "symfony/yaml": ">=2.7 <6.0"
                "symfony/yaml": "^4.4.20 || ^5.0"
                "yaml"
                "symfony/yaml": "^5.4 || ^6.0 || ^7.0",
                "symfony/yaml": "Required if you'd like to use the YAML metadata format."
            "name": "symfony/yaml",
                "url": "https://github.com/symfony/yaml.git",
                "url": "https://api.github.com/repos/symfony/yaml/zipball/c441e9d2e340642ac8b951b753dea962d55b669d",
                "Resources/bin/yaml-lint"
                "source": "https://github.com/symfony/yaml/tree/v5.3.14"
fab@MBP14-Fab-2 /t/244mirror> cat ../244/composer.lock |grep yaml
                "ext-yaml": "*",
                "symfony/yaml": "^4.4|^5.0|^6.0"
                "symfony/yaml": "To use the yaml reference dumper"
                "symfony/yaml": "<4.4.26"
                "symfony/yaml": "^4.4.26|^5.0|^6.0"
                "symfony/yaml": ""
                    "generated/yaml.php",
                "symfony/yaml": "~3|~4|~5|~6|~7"
                "symfony/yaml": "If you want to parse features, represented in YAML files"
                "symfony/yaml": ">=2.7 <6.0"
                "symfony/yaml": "^4.4.20 || ^5.0"
                "yaml"
                "symfony/yaml": "^5.4 || ^6.0 || ^7.0",
                "symfony/yaml": "Required if you'd like to use the YAML metadata format."
            "name": "symfony/yaml",
                "url": "https://github.com/symfony/yaml.git",
                "url": "https://api.github.com/repos/symfony/yaml/zipball/c441e9d2e340642ac8b951b753dea962d55b669d",
                "Resources/bin/yaml-lint"
                "source": "https://github.com/symfony/yaml/tree/v5.3.14"

same goes for deprecation contracts

fab@MBP14-Fab-2 /t/244mirror> cat composer.lock |grep deprecation-contracts
                "symfony/deprecation-contracts": "^2.2 || ^3.0"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
            "name": "symfony/deprecation-contracts",
                "url": "https://github.com/symfony/deprecation-contracts.git",
                "url": "https://api.github.com/repos/symfony/deprecation-contracts/zipball/80d075412b557d41002320b96a096ca65aa2c98d",
                "source": "https://github.com/symfony/deprecation-contracts/tree/v2.5.3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1",
fab@MBP14-Fab-2 /t/244mirror> cat ../244/composer.lock |grep deprecation-contracts
                "symfony/deprecation-contracts": "^2.2 || ^3.0"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
            "name": "symfony/deprecation-contracts",
                "url": "https://github.com/symfony/deprecation-contracts.git",
                "url": "https://api.github.com/repos/symfony/deprecation-contracts/zipball/80d075412b557d41002320b96a096ca65aa2c98d",
                "source": "https://github.com/symfony/deprecation-contracts/tree/v2.5.3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1",

I installed both with php7.4 with

php7 /opt/homebrew/bin/composer create-project --repository-url=https://mirror.mage-os.org/ magento/project-community-edition:2.4.4-p11 .
php7 /opt/homebrew/bin/composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition:2.4.4-p11 .

symfony packages are know to be installed in different versions depending on the php version in use, maybe that could be the source of the problem?

@fballiano
Copy link
Contributor Author

indeed, reinstalling both with php8.1 generates the difference

fab@MBP14-Fab-2 /tmp> cat 244/composer.lock|grep yaml
                "ext-yaml": "*",
                "symfony/yaml": "^4.4|^5.0|^6.0"
                "symfony/yaml": "To use the yaml reference dumper"
                "symfony/yaml": "<4.4.26"
                "symfony/yaml": "^4.4.26|^5.0|^6.0"
                "symfony/yaml": ""
                    "generated/yaml.php",
                "symfony/yaml": "~3|~4|~5|~6|~7"
                "symfony/yaml": "If you want to parse features, represented in YAML files"
                "symfony/yaml": ">=2.7 <6.0"
                "symfony/yaml": "^4.4.20 || ^5.0"
                "yaml"
                "symfony/yaml": "^5.4 || ^6.0 || ^7.0",
                "symfony/yaml": "Required if you'd like to use the YAML metadata format."
            "name": "symfony/yaml",
                "url": "https://github.com/symfony/yaml.git",
                "url": "https://api.github.com/repos/symfony/yaml/zipball/c441e9d2e340642ac8b951b753dea962d55b669d",
                "Resources/bin/yaml-lint"
                "source": "https://github.com/symfony/yaml/tree/v5.3.14"


fab@MBP14-Fab-2 /tmp> cat 244mirror/composer.lock |grep yaml
                "ext-yaml": "*",
                "symfony/yaml": "^4.4|^5.0|^6.0"
                "symfony/yaml": "To use the yaml reference dumper"
                "symfony/yaml": "<4.4.26"
                "symfony/yaml": "^4.4.26|^5.0|^6.0"
                "symfony/yaml": ""
                    "generated/yaml.php",
                "symfony/yaml": "~3|~4|~5|~6|~7"
                "symfony/yaml": "If you want to parse features, represented in YAML files"
                "symfony/yaml": ">=2.7 <6.0"
                "symfony/yaml": "^4.4.20 || ^5.0"
                "yaml"
                "symfony/yaml": "^5.4 || ^6.0 || ^7.0",
                "symfony/yaml": "Required if you'd like to use the YAML metadata format."
            "name": "symfony/yaml",
                "url": "https://github.com/symfony/yaml.git",
                "url": "https://api.github.com/repos/symfony/yaml/zipball/29b60e88ff11a45b708115004fdeacab1ee3dd5d",
                "source": "https://github.com/symfony/yaml/tree/5.0"

@rhoerr
Copy link
Contributor

rhoerr commented Nov 13, 2024

Relevant (not merged): mage-os/github-actions#231

Also, I don't think this would have caused it since it's specific to the unit-tests workflow, but also related to actions PHP version: mage-os/infrastructure#45

@rhoerr
Copy link
Contributor

rhoerr commented Nov 13, 2024

Ultimately I suppose this probably isn't a change in our actions at all, but possibly a difference in our constraints and Symfony releasing updates with higher PHP requirement.

@rhoerr
Copy link
Contributor

rhoerr commented Nov 13, 2024

@fballiano What is the output of composer why symfony/yaml on both?

@fballiano
Copy link
Contributor Author

@rhoerr I was checking exactly that

fab@MBP14-Fab-2 /t/244> /opt/homebrew/opt/[email protected]/bin/php /opt/homebrew/bin/composer why symfony/yaml
codeception/codeception      4.2.2   requires  symfony/yaml (>=2.7 <6.0) 
symfony/dependency-injection v5.4.45 conflicts symfony/yaml (<4.4.26)    

fab@MBP14-Fab-2 /t/244mirror> /opt/homebrew/opt/[email protected]/bin/php /opt/homebrew/bin/composer why symfony/yaml
codeception/codeception      4.2.2   requires  symfony/yaml (>=2.7 <6.0) 
symfony/dependency-injection v5.4.45 conflicts symfony/yaml (<4.4.26)

but most importantly

fab@MBP14-Fab-2 /t/244> /opt/homebrew/opt/[email protected]/bin/php /opt/homebrew/bin/composer prohibits symfony/yaml 5.3.14
There is no installed package depending on "symfony/yaml" in versions not matching 5.3.14
Not finding what you were looking for? Try calling `composer update "symfony/yaml:5.3.14" --dry-run` to get another view on the problem.

fab@MBP14-Fab-2 /t/244mirror> /opt/homebrew/opt/[email protected]/bin/php /opt/homebrew/bin/composer prohibits symfony/yaml 5.3.14
symfony/yaml                      v5.3.14   requires         symfony/deprecation-contracts (^2.1)                    
magento/project-community-edition 2.4.4-p11 does not require symfony/deprecation-contracts (but v3.5.0 is installed) 
Not finding what you were looking for? Try calling `composer update "symfony/yaml:5.3.14" --dry-run` to get another view on the problem.

@fballiano
Copy link
Contributor Author

checking deprecation contracts

fab@MBP14-Fab-2 /tmp> cat 244/composer.lock|grep deprecation-contracts
                "symfony/deprecation-contracts": "^2.2 || ^3.0"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
            "name": "symfony/deprecation-contracts",
                "url": "https://github.com/symfony/deprecation-contracts.git",
                "url": "https://api.github.com/repos/symfony/deprecation-contracts/zipball/80d075412b557d41002320b96a096ca65aa2c98d",
                "source": "https://github.com/symfony/deprecation-contracts/tree/v2.5.3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1",

fab@MBP14-Fab-2 /tmp> cat 244mirror/composer.lock |grep deprecation-contracts
                "symfony/deprecation-contracts": "^2.2 || ^3.0"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
            "name": "symfony/deprecation-contracts",
                "url": "https://github.com/symfony/deprecation-contracts.git",
                "url": "https://api.github.com/repos/symfony/deprecation-contracts/zipball/0e0d29ce1f20deffb4ab1b016a7257c4f1e789a1",
                "source": "https://github.com/symfony/deprecation-contracts/tree/v3.5.0"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3"
                "symfony/deprecation-contracts": "^2.1|^3",
                "symfony/deprecation-contracts": "^2.1|^3",

we have https://github.com/symfony/deprecation-contracts/tree/v2.5.3 for upstream and https://github.com/symfony/deprecation-contracts/tree/v3.5.0 for mirror

@fballiano fballiano mentioned this issue Nov 21, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Workaround for 2.4.4-p* integrity check fballiano/mageos-generate-mirror-repo-js
2 participants
@fballiano @rhoerr