[FEATURE] Replace poetry with uv for project management

[NTFSvolume]

Describe the feature you'd like

uv is a project management utility for Python that includes functionality to replace pip, pipx, poetry, virtualenv, and more.

While using poetry is fine since we don't have any special development requirements (like extras or private dependencies), switching offers significant advantages for the users, and provides some minor but nice to have improvements for development.

Pros of using uv

1. Remove the requirement of having Python installed to run CDL

Although uv can be installed with pip, they also provide a standalone binary for direct installation beacuse the are actually written in Rust, not Python. This means uv can be installed without Python. Additionally, uv itself can install Python. That means we can update the start scripts to basically do the following:

1. Verify if uv is installed on the system. If not, use irm | iex or curl to download and execute the installer. No admin or sudo privileges are needed.
2. Check if a compatible Python version is available on the system. If not, install it (uv install python 3.11).
3. Check if cyberdrop-dl-patched is available. If not, install it (uv tool install cyberdrop-dl-patched).
4. Run cyberdrop-dl-patched (uv tool run cyberdrop-dl-patched).

I added steps 2 and 3 for clarity, but they are actually not needed. uv will automatically install Python (if required) and then install & run cyberdrop-dl-patched with step 4.

2. Virtual environment creation and activation will be managed automatically (for the users).

uv will also make cyberdrop-dl available globally (user scope) without having to activate a virtual environment.

3. Switch the pyproject.toml format to the Python standard [project] section (PEP 621) instead of the poetry specific [tool.poetry].

This will make the project information available to other tools. poetry will switch to this format as well in the upcoming v2.0.0, so these changes will actually be required at some point.

4. uv is fast, like... really fast. Way faster than poetry

Installing uv, Python and CDL from scratch takes less than 15 seconds in my case on a fresh VM.

uv run is fast enough that every invocation will implicitly re-lock and re-sync the project,
ensuring that your environment is up-to-date without the need for manual intervention.

source: https://astral.sh/blog/uv-unified-python-packaging#projects

poetry also have a bug (which is over 4 years old at this point...) where it will hang forever while trying to install dependencies if you are executing it on a machine without a DE (linux) because it always tries to use keyrings (see: GH repo python-poetry/poetry, issue #8623). I run into this all the time and have to manually set my keyring to null before running any poetry command

Cons of using uv

1. uv does not have their own build back-end (yet).

They recommend hatchling by default but we can still use poetry-core for the builds. uv will manage it automatically like any another dependency

2. uv is technically still in beta.

Current version is v0.5.13. However, it is production ready and it is under really active development. They are less that a year old but already have more GH stars that poetry

3. We may need to upload the start scripts individually instead of zipping them together

This is to make sure they don't trigger any Boxter warning on Windows. This is inconvenient cause the AppData folder will be created in the same folder as the script but there will be no indicative that it belongs to CDL cause there will be no parent Cyberdrop-DL vX folder.

I can think of a few ways we can solve this from with the start script itself. I will look into it if this gets approved

Describe alternatives you've considered

No response

Additional context

The main benefit of the switch will be the first item on the pros. We will be able to release a single, self-contained installer that does not require any previous preparation from the user, without the drawbacks of releasing a compiled .appimage or .exe (ex: Trojan false positives from antivirus).

This benefit is primarily for the "normal" users: the ones that use the start scripts. Obviously CDL will still be available for installation via normal pip or pipx.

See: "uv: Unified Python packaging" for a more detailed break down of the advantages of using uv for project management

Tasks required to migrate

• Modify the pyproject.toml file to comply with the PEP 621 format
• Add a new script entry point in the project to ensure CDL can be run with both cyberdrop-dl and cyberdrop-dl-patched, to simplify program calls with uv.
• Update start scripts.
• Update publish to PyPi action.
• Update contributing guidelines
• Update install instructions on wiki

Alternative

Another option will be to use both: Use uv for the start scripts but still use poetry for the project management.

I can make the PR with the necessary changes (if any)

[baccccccc]

I know virtually nothing about Python ecosystem, but just wanted to point out that this

Check if a compatible Python version is available on the system. If not, install it (uv install python 3.11).

can also be achieved with whatever package manager is native to the platform. E.g., winget if running on Windows, etc. (This also does not require local admin rights if you use packaged Python installation from Microsoft Store. E.g., winget.exe install 9PNRBTZXMB4Z, as I do.)

And hence you can satisfy this requirement

1. Remove the requirement of having Python installed to run CDL

simply by adding a line or two into existing startup scripts. You don't have to introduce additional dependencies or revamp how the project is managed today.

(Again, I'm not downplaying other potential benefits. Just pointing out that if satisfying Python dependency is the primary challenge today, it can be addressed somewhat easier.)

[NTFSvolume]

I use winget too but it is pretty unreliable.

1. It's available since windows 10 but not on all of them. It was added on Windows 10 1709
2. The CLI interface is not enabled by default. It requires the user to have opened the Microsoft store at least once cause it requires registration
3. Even when enabled, the path that it uses to save the executables is created at random and not garantied to be added to the PATH cause that functinality is not implemented at all by winget. It's up to the packager to implement the logic to add the installation path to the system PATH.

---

We can't solve 1, there will be users that will just not have winget installed
We can solve 2 by triggering a manual registration but we should have to
We can solve 3 by running winget with --scope machine to make sure that installation path is absolute and we know before hand what it will be, but that requires running as admin which is another problem.

Ultimately is not a good robust solution. Fetching and executing the uv binary is much simpler and is guarantied to always work

[NTFSvolume]

For context: microsoft/winget-cli#549

[baccccccc]

yes, this is why, instead of “one line,” I said “one or two lines.” And maybe that was an exaggeration :)

of course, it's hard to make blatant statements like ”winget solves everything for every piece of software” because every piece of software is different. But if we're only talking about one specific dependency (Python package) it's more or less easy to make it reliable.

1. Always install from Microsoft store, this makes it per-user (i.e., no system-wide permissions required.)
2. Fetch the full binary path which is relative to user's profile and also depends on the package name.

the package name itself is constant since you have to know it in advance in order to install it. In other words, the only situation when it changes is if you want to migrate to a newer version of Python. And hence the package name is new, and the full binary path is new as well. But in this case, you'd be updating the startup scripts anyway (even with uv if I'm not mistaken.)

I'm not saying there's no work whatsoever, but effectively it boils down to

$packageCode = '9PNRBTZXMB4Z'         # Python 3.13
$packageName = 'PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0'
$pythonName  = 'Python313'

& winget.exe install $packageCode
& "$env:localAppData\Microsoft\WindowsApps\$packageName\pip.exe" install --upgrade 'cyberdrop-dl-patched' --no-warn-script-location
& "$env:localAppData\Microsoft\WindowsApps\$packageName\python.exe" "$env:localAppData\Packages\$packageName\LocalCache\local-packages\$pythonName\site-packages\cyberdrop_dl\main.py"

and you can still run this via irm | iex although I'd expect any decent antivirus to freak out at this point. But as long as you just publish it as startup script and make user download and run it manually, you should be golden.

this way, you don't even have to mess with user's %path% value which might have unintended consequences. Let alone drop yet another binary on user's machine which you're somehow now responsible for :)

of course you might want to make it slightly nicer, e.g. by swallowing the output (which might be confusing to the end user), suppressing some prompts and maybe adding rudimentary error handling.

but to me it still sounds like a lot less work than what you described is needed to move to a different project management tool. And these changes are only limited to startup scripts anyway, i.e. the rest of the project will remain intact.

with that said, I cannot argue about winget availability. I.e. users on older OS versions (and also on LTSCs) are out of luck with this approach. But IMO that's acceptable since you're going to keep the old ways to run CDL anyways. So whoever cannot use winget for some reason can continue to use old manual ways to install Python and run CDL. And I think it's reasonable to expect most people to be on a relatively modern OS version if they value convenience.

also the note about “having to open Microsoft store at least once” applies only to fresly installed OS where the user has never logged in before. How likely it is that someone just instaled Windows, and then they want to run CDL immediately? I don't think that's a mainstream scenario we should optimize for.

moreover, if someone finds themselves in this situation, they probably already know what they're doing. E.g., they deploy a brand new VM just for CDL, this implies they know how to install and run CDL already.

[NTFSvolume]

Interesting. I don't like idea of having harcoded paths but pinning the package to an specific version seems good enough to me. I couldn't find any documentation about it but i would expect the path to not change after an update

As a drawback, this means that the user needs to have that specific version of python and it must be from the store, not from the python page. Not ideal but honestly, i think is fine cause the script is just for convenience.

I adapted the code you provided to plain batch + added the venv stuff. No complains from virus total, either from the bare script nor the zipped file

@echo off
setlocal enabledelayedexpansion
chcp 65001

rem User defined variables

set "VENV_DIR="
set "COMMANDLINE_ARGS="

rem ----------------------------------------------------------

set "cdl_version=>=6.0,<7.0"
set "package_id=9PNRBTZXMB4Z"
set "package_name=PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0"
set "python_loc=%localappdata%\Microsoft\WindowsApps\%package_name%\python.exe"

echo Checking Python install ... && winget list %package_id% >nul || call:InstallPython || pause && exit /b 1
call:PrepareVenv || pause && exit /b 1

echo Installing / Updating Cyberdrop-DL...
pip uninstall -y -qq cyberdrop-dl
cls && pip install "cyberdrop-dl-patched%cdl_version%" --upgrade --no-warn-script-location --no-compile
cyberdrop-dl %COMMANDLINE_ARGS% 
pause
exit /b

:PrepareVenv
if not defined VENV_DIR (set "VENV_DIR=%~dp0%venv")
if not exist "%VENV_DIR%" (mkdir "%VENV_DIR%")
call:CreateVenv || exit /b 1
echo Attempting to start venv ...
call "%VENV_DIR%\Scripts\activate.bat"
exit /b

:CreateVenv
if not exist "%VENV_DIR%\Scripts\activate.bat" (
    echo Creating virtual environment ..
    "%python_loc%" -m venv "%VENV_DIR%"
)
exit /b

:InstallPython
echo Installing Python 
winget install -e --id %package_id% 
exit /b

However, this can't be replicated on macOS or Linux cause macOS does not have any default package manager and linux have some_many that is not worth adding the check and conditions for all of them.

We can curl and install homebrew on macOS but at that point we are better of installing uv cause then the exact same code can be used for linux. These OSs do not complain about possible malware

#!/bin/sh
# User defined variables
COMMANDLINE_ARGS=""

# ----------------------------------------------------------
cdl_version=">=6.0.0,<7.0.0"
if ! which uv > /dev/null 2>&1; then
    curl -LsSf https://astral.sh/uv/install.sh | sh || exit 1
fi

clear && uvx --python 3.13 --from "cyberdrop-dl-patched${cdl_version}" --refresh --no-build cyberdrop-dl $COMMANDLINE_ARGS

with that said, I cannot argue about winget availability. I.e. users on older OS versions (and also on LTSCs) are out of luck with this approach. But IMO that's acceptable since you're going to keep the old ways to run CDL anyways. So whoever cannot use winget for some reason can continue to use old manual ways to install Python and run CDL. And I think it's reasonable to expect most people to be on a relatively modern OS version if they value convenience.

I think this is reasonable as well

but to me it still sounds like a lot less work than what you described is needed to move to a different project management tool. And these changes are only limited to startup scripts anyway, i.e. the rest of the project will remain intact.

We don't have to switch uv. We can apply the uv or winget changes to the scripts but still use poetry for the repo. However, i do think switching to uv on the repo is worth it, specially on the long term. However, it's not up to me to decide this kind of changes to the repo and that's why i opened this issue.

I do like the snippet you provided with widget though. Thanks for the input

[baccccccc]

Interesting. I don't like idea of having harcoded paths but pinning the package to an specific version seems good enough to me. I couldn't find any documentation about it but i would expect the path to not change after an update

yeah. The way Python maintainers decided to manifest their packages is quite unusual. Each minor version gets its own app published in the Store. For example:

• https://apps.microsoft.com/detail/9NRWMJP3717K
• https://apps.microsoft.com/detail/9NCVDN91XZQP
• https://apps.microsoft.com/detail/9PNRBTZXMB4Z

each of these has its own package code (and name), as can be seen from the links. And therefore, once installed, each has its own path, so that they can be installed side-by-side.

but when the patch version changes (e.g. from 3.13.240 to 3.13.496) then of course it's just an update to the same app, and the ID and path won't churn. Hence it just updates in place, and normally the user won't even notice anything. (Their data in %LocalAppData% remains intact.)

therefore I find the need to hardcode the path only mildly infuriating. But it is not as bad as hardcoding usually is. This path will only change once Python minor (or major) version changes. (E.g., from 3.13 to 3.14.) This will require us to churn both package code and the path in lockstep. But this will require updating the startup scripts anyway, assuming we put the package code in there.