Can't export virtualenv for coverage-py or mypy-extra-type-stubs

[jensenja]

Describe the bug
I'm unable to pants export a virtualenv for coverage-py or mypy-extra-type-stubs. Attempting either gets me the same error as below:

root@archdev:~/pants-testing# ./pants export --py-resolve-format=symlinked_immutable_virtualenv --resolve=coverage-py
22:58:15.37 [ERROR] 1 Exception encountered:

Engine traceback:
  in `export` goal

UnrecognizedResolveNamesError: Unrecognized resolve name from the option --export-resolve: coverage-py

All valid resolve names: ['bandit', 'black', 'coverage-py', 'debugpy', 'docformatter', 'flake8', 'ipython', 'isort', 'lambdex', 'mypy', 'mypy-extra-type-stubs', 'pytest', 'python-default', 'setuptools']

Pants version
2.16.0

OS
Linux. Have not tried macOS. See additional info

Additional info
The base system is Arch and I'm running a custom development container that's using pants/poetry/pyenv with Debian Bookworm as a base. Dockerfile contents below:

FROM debian:bookworm

ENV DEBIAN_FRONTEND=noninteractive LANG=en_US.UTF-8 LC_ALL=C.UTF-8 LANGUAGE=en_US.UTF-8

RUN apt-get -q update && apt-get -qqy upgrade && apt-get -qqy dist-upgrade

RUN apt-get -qqy install \
    build-essential \
    ca-certificates \
    git \
    libssl-dev \
    zlib1g-dev \
    libbz2-dev \
    libreadline-dev \
    libsqlite3-dev \
    wget \
    curl \
    llvm \
    unzip \
    tree \
    libncurses5-dev \
    xz-utils \
    tk-dev \
    libxml2-dev \
    libxmlsec1-dev \
    libffi-dev \
    liblzma-dev \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ENV HOME=/root
ENV PYENV_ROOT=$HOME/.pyenv
ENV PATH=$PYENV_ROOT/shims:$PYENV_ROOT/bin:$HOME/.local/bin:$HOME/bin:$PATH

RUN echo "installing pyenv" \
    && curl https://pyenv.run | bash \
    && echo "installing Python 3.11.4" \
    && pyenv install 3.11.4 \
    && echo "setting global Python version to 3.11.4" \
    && pyenv global 3.11.4 \
    && echo "installing poetry" \
    && curl -sSL https://install.python-poetry.org | python3 - \
    && poetry config virtualenvs.in-project true \
    && echo "installing pants" \
    && curl --proto '=https' --tlsv1.2 -fsSL https://static.pantsbuild.org/setup/get-pants.sh | bash

WORKDIR /root

Once I spin up this container I then git clone a repository that has the pants binary included in it, then run my pants goals from the git repository folder using ./pants fmt :: ./pants export etc etc

[WorkerPants]

Welcome to the Pantsbuild Community. This looks like your first issue here. Thanks for taking the time to write it.

If you haven't already, feel free to come say hi on Slack.

If you have questions, or just want to surface this issue, check out the #development channel.
(If you want to check it out without logging in, check out our Linen mirror)

Thanks again, and we look forward to your next Issue/PR 😄!

[benjyw]

uuuuuuugh, I suspect this is due to dashes in the resolve names...

[jensenja]

@benjyw i initially thought that as well, but i can export my python-default resolve without issue. 😕

[benjyw]

Does this persist in 2.17.0rc3, say?

[benjyw]

Note that post-2.16.0 those special tool lockfiles aren't a thing any more, and the only lockfiles are the ones you explicitly enumerate in [python-setup].resolves. So you won't be able to export the Pants default versions without extra work.

[jensenja]

@benjyw understood - do i take this to mean that pants will no longer include pytest/mypy/etc and it will be up to the user to wire up specific tools+versions to tell pants [fmt|check|lint|test] what to do? my main concern for the tool lockfiles was being able to tell VSCode what to do

EDIT: if this is the case then I won't mind closing this as wontfix

[benjyw]

Pants will still include default versions of those, so things will continue to work out of the box. But you won't be able to export them unless you explicitly configure them. Which I can see is a hassle for your specific use case. Hmm.

[benjyw]

So you can still fmt/check/lint/test without configuring anything, as default lockfiles for those are still provided, but they aren't named resolves that you can export.

[benjyw]

But the use case of "I want to export the default mypy lockfile so vscode can use it" is not covered, true. I will have a think about that one.

[jensenja]

cheers thanks @benjyw - let me know how I can help!

[benjyw]

Ah, so, one solution is, for each tool you want to export, add this explicitly to your pants.toml, under [python.resolves]:

isort = "resource://pants.backend.python.lint.isort/isort.lock"

(pointing to the right package and lockfile for that tool, obviously)

That maps the default lockfile to a named resolve, so you can then export it.

[benjyw]

This should all work in 2.16.x I think, but certainly in 2.17.x