ModuleNotFoundError: No module named 'pip' when adding ksql

[fferegrino]

• Poetry version: 1.2.2 (though it happens in versions as early as 1.1.8)
• Python version: 3.8.11
• OS version and name: macOS 12.6
• pyproject.toml: https://gist.github.com/fferegrino/4dc43d3e33a645f01a1b332e5ee2ec7c

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Poetry fails when installing the ksql package (poetry add ksql) with the following error:

  EnvCommandError

  Command ['/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8', '--no-deps', '/Users/antonio.feregrino/Library/Caches/pypoetry/artifacts/21/c7/72/9ca32f9f222d0e2147d8d8ea233fa908c430ef8e8ef8c1e32982fb0f22/ksql-0.10.2.tar.gz'] errored with the following return code 1, and output:
  Processing /Users/antonio.feregrino/Library/Caches/pypoetry/artifacts/21/c7/72/9ca32f9f222d0e2147d8d8ea233fa908c430ef8e8ef8c1e32982fb0f22/ksql-0.10.2.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [17 lines of output]
        Traceback (most recent call last):
          File "/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
            main()
          File "/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/private/var/folders/26/kknv0qdn23x8cn86y7p06j4r0000gp/T/pip-build-env-df0a3emt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/private/var/folders/26/kknv0qdn23x8cn86y7p06j4r0000gp/T/pip-build-env-df0a3emt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
            self.run_setup()
          File "/private/var/folders/26/kknv0qdn23x8cn86y7p06j4r0000gp/T/pip-build-env-df0a3emt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 483, in run_setup
            super(_BuildMetaLegacyBackend,
          File "/private/var/folders/26/kknv0qdn23x8cn86y7p06j4r0000gp/T/pip-build-env-df0a3emt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
            exec(code, locals())
          File "<string>", line 8, in <module>
        ModuleNotFoundError: No module named 'pip'
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

Now, I know it seems like the issue is with PIP, since when I execute the command:

/Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8/bin/python -m pip install --use-pep517 --disable-pip-version-check --prefix /Users/antonio.feregrino/Library/Caches/pypoetry/virtualenvs/th-OUeCrz7m-py3.8 --no-deps /Users/antonio.feregrino/Library/Caches/pypoetry/artifacts/21/c7/72/9ca32f9f222d0e2147d8d8ea233fa908c430ef8e8ef8c1e32982fb0f22/ksql-0.10.2.tar.gz

The same error shows.

However, when executing a traditional pip install ksql it installs without any issue.

Now my question is, why does Poetry try to install from its caches and not reproduce the behaviour of plain pip?

[neersighted]

Poetry is not pip, and much of Poetry's richness comes from it not being pip. As you can see, you can reproduce the error with the command Poetry uses, and it's not actually Poetry or Pip's fault.

Instead, the project you are trying to install has an undeclared build-time dependency on pip, and needs to declare it as such:

https://github.com/bryanyang0528/ksql-python/blob/6161b481b9c872a0693f30cede5b46ae23fd2336/setup.py#L10-L13

Typically this is solved with the deprecated setup_requires argument to setup(), or use of build-system.requires in pyproject.toml. By default, packages that do not declare a build-system get setuptools and nothing else. To use a non-setuptools build system, or to use more than just the stdlib+setuptools, PEP 517-compliant build tools (like Poetry) require those dependencies to be declared.

Let me know if anything is unclear and I can dig up similar issues to show you how it was resolved with other projects.

[neersighted]

I created bryanyang0528/ksql-python#117 for you!

[fferegrino]

Thank you! I appreciate the time you took on addressing my issue.

I suppose, my follow up question would be if there is any recommendations on what to do or work arounds before the issue gets fixed on their end.

Anyways, thanks again.

[neersighted]

The trivial fix is to create a temporary fork and depend on that with a URL or VCS dependency. You can even send a patch upstream, and depend on your pull request branch, if you want.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.