Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

info: fix metadata build error propagation #9870

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

abn
Copy link
Member

@abn abn commented Nov 24, 2024

This change fixes a regression that caused the swallowing of PEP517 build errors when isolated builds fail during metadata build.

Borrowing from another issue, you can use the following pyproject.toml to reproduce/test.

[tool.poetry]
name = "my-package"
version = "0.1.0"
description = ""
authors = ["My Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
mmagic = {git = "https://github.com/open-mmlab/mmagic.git", rev = "0a560bb"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Output w/ Poetry (1.8.4)

Updating dependencies
Resolving dependencies... (3.1s)

Unable to determine package info for path: /tmp/foo/.venv/src/mmagic

Command ['/tmp/tmp_xptxh3p/.venv/bin/python', '-I', '-W', 'ignore', '-c', "import build\nimport build.env\nimport pyproject_hooks\n\nsource = '/tmp/foo/.venv/src/mmagic'\ndest = '/tmp/tmp_xptxh3p/dist'\n\nwith build.env.DefaultIsolatedEnv() as env:\n    builder = build.ProjectBuilder.from_isolated_env(\n        env, source, runner=pyproject_hooks.quiet_subprocess_runner\n    )\n    env.install(builder.build_system_requires)\n    env.install(builder.get_requires_for_build('wheel'))\n    builder.metadata_path(dest)\n"] errored with the following return code 1

Error output:
Traceback (most recent call last):
  File "<string>", line 13, in <module>
    env.install(builder.get_requires_for_build('wheel'))
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/tmp_xptxh3p/.venv/lib/python3.13/site-packages/build/__init__.py", line 239, in get_requires_for_build
    with self._handle_backend(hook_name):
         ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/lib64/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/tmp/tmp_xptxh3p/.venv/lib/python3.13/site-packages/build/__init__.py", line 360, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

Fallback egg_info generation failed.

Command ['/tmp/tmp_xptxh3p/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1

Output:
Traceback (most recent call last):
  File "/tmp/foo/.venv/src/mmagic/setup.py", line 9, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

Output w/ Poetry (git@main)

Updating dependencies
Resolving dependencies... (0.4s)

Unable to determine package info for path: /tmp/foo/.venv/src/mmagic

Backend subprocess exited when trying to invoke get_requires_for_build_wheel

PEP517 build failed

Output w/ PR

Updating dependencies
Resolving dependencies... (0.3s)

Unable to determine package info for path: /tmp/foo/.venv/src/mmagic


    | Command '['/tmp/tmp7lntsszc/.venv/bin/python', '/home/abn/workspace/python-poetry/poetry/.venv/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmpc3pwlc6d']' returned non-zero exit status 1.
    | 
    | Traceback (most recent call last):
    |   File "/home/abn/workspace/python-poetry/poetry/.venv/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    |     main()
    |     ~~~~^^
    |   File "/home/abn/workspace/python-poetry/poetry/.venv/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    |     json_out["return_val"] = hook(**hook_input["kwargs"])
    |                              ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/home/abn/workspace/python-poetry/poetry/.venv/lib64/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 134, in get_requires_for_build_wheel
    |     return hook(config_settings)
    |   File "/tmp/tmp7lntsszc/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
    |     return self._get_build_requires(config_settings, requirements=[])
    |            ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/tmp/tmp7lntsszc/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
    |     self.run_setup()
    |     ~~~~~~~~~~~~~~^^
    |   File "/tmp/tmp7lntsszc/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup
    |     super().run_setup(setup_script=setup_script)
    |     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/tmp/tmp7lntsszc/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
    |     exec(code, locals())
    |     ~~~~^^^^^^^^^^^^^^^^
    |   File "<string>", line 9, in <module>
    | ModuleNotFoundError: No module named 'torch'
    | 

Note: This error originates from the build backend, and is likely not a problem with poetry but with the package at /tmp/foo/.venv/src/mmagic

  (a) not supporting PEP 517 builds
  (b) not specifying PEP 517 build requirements correctly; or
  (c) the build requirement not being successfully installed in your system environment.

You can verify this by running pip wheel --no-cache-dir --use-pep517 "/tmp/foo/.venv/src/mmagic".

You can test this pull request using the following command. You can swap podman with docker if you use that.

podman run --rm -i --entrypoint bash docker.io/python:3.13 <<EOF
set -xe
python -m pip install --root-user-action ignore  --disable-pip-version-check -q git+https://github.com/python-poetry/poetry.git@refs/pull/9870/head
install -d foobar
pushd foobar
cat > pyproject.toml <<TOML
[tool.poetry]
name = "my-package"
version = "0.1.0"
description = ""
authors = ["My Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
mmagic = {git = "https://github.com/open-mmlab/mmagic.git", rev = "0a560bb"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
TOML
poetry install
EOF

This change fixes a regression that caused the swallowing of PEP517
build errors when isolated builds fail during metadata build.
@abn abn force-pushed the propagate-build-metadata-error branch from e62c95a to c305c6d Compare November 24, 2024 00:45
@abn abn requested a review from a team November 24, 2024 00:46
@abn abn marked this pull request as ready for review November 24, 2024 00:47
@abn abn added the area/error-handling Bad error messages/insufficient error handling label Nov 24, 2024
Copy link
Member

@radoering radoering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Have you thought about adding the indentation of the inner error and/or the listing of possible reasons also at

"<info>"
"Note: This error originates from the build backend,"
" and is likely not a problem with poetry"
f" but with {pkg.pretty_name} ({pkg.full_pretty_version})"
" not supporting PEP 517 builds. You can verify this by"
f" running '{pip_command} \"{requirement}\"'."
"</info>"
?

@abn
Copy link
Member Author

abn commented Nov 24, 2024

Yeah. But figured I'll do that in a separate PR just to keep things clean here. I also wanted to see if we can unify that somehow. Once this is accepted I can work on that, ie. No one has objections to the formatting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/error-handling Bad error messages/insufficient error handling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants