ran pre-commit

.github/ISSUE_TEMPLATE/bug_report.md

@@ -24,7 +24,7 @@ If applicable, add screenshots to help explain your problem.
  - Cyberdrop Downloader Version #
 
 **Downloader.log file**
-You need to include the downloader.log file after the issue occurs. Majority of the time it's the only way to diagnose issues. 
+You need to include the downloader.log file after the issue occurs. Majority of the time it's the only way to diagnose issues.
 If you open a bug report and don't include the log file, it will be closed.
 
 **Additional context**

.gitignore

@@ -20,4 +20,4 @@ venv
 /dist/
 
 # MacOS system files
-**.DS_Store
+**.DS_Store

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v5.0.0  
+    rev: v5.0.0
     hooks:
       - id: trailing-whitespace
         args: [--markdown-linebreak-ext=md]

CHANGELOG.md

@@ -13,7 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 This update introduces the following changes:
 
-1. Adds the option to limit how many items are scraped 
+1. Adds the option to limit how many items are scraped
 2. Add support for scraping a users' coomer favorites
 3. Add integration to handle downloads supported by https://real-debrid.com
 4. Add support for https://nekohouse.su profiles and posts
@@ -29,7 +29,7 @@ This update introduces the following changes:
 
 - Users can limit the number of items to scrape by type, using the `--maximum-number-of-children` parameter. For more details on how to use this feature, visit the wiki: https://script-ware.gitbook.io/cyberdrop-dl/reference/configuration-options/settings#download-options
 - Add support for scraping a users' coomer favorites by allowing the user to pass the coomer favorites page URL as an input URL (https://coomer.su/favorites). This requires them to have their coomer session token in the `authentication.yaml` file.
-- Add real-debrid integration to download from any site that they support (`mega.nz`,`rapidgator`, `google drive`, `k2s`, etc). User needs to provide their API key in the `authentication.yaml` file in order to allow downloads 
+- Add real-debrid integration to download from any site that they support (`mega.nz`,`rapidgator`, `google drive`, `k2s`, etc). User needs to provide their API key in the `authentication.yaml` file in order to allow downloads
 - Nekohouse URLs can now be scraped and downloaded by CDL
 - Users can now get the stats report of the run via multiple services and include the main log as an attachment. For more information on how to setup notifications, visit: https://script-ware.gitbook.io/cyberdrop-dl/reference/notifications
 - Fix parsing of bunkr file extensions when `--remove-generated-id` is enabled
@@ -41,7 +41,7 @@ This update introduces the following changes:
 ## [5.7.0] - 2024-10-25
 
 This update introduces the following changes:
-1. Rotating log files 
+1. Rotating log files
 2. Overhaul hashing functions
 3. Add support for https://tokyomotion.net URLs
 4. Add support for https://xxxbunker.com URLs
@@ -69,7 +69,7 @@ This update introduces the following changes:
 - Fix error during program exit when referers table no longer exists
 - Prevents crashes when there are insufficient permissions to move a file
 - Fix an issue where CDL would delete URLs input file
-- Move functions for after download to `post_runtime` 
+- Move functions for after download to `post_runtime`
 - Fix handling of deleted imgbox albums if they return `HTTP 200`
 - Fix cyberfile deleted folders not being correctly scraped
 - Update logging to include when a file is being downloaded with no crawler
@@ -164,7 +164,7 @@ This update introduces the following changes:
 ## [5.6.40] - 2024-10-01
 
 This update introduces the following changes:
-1. Fixes empty folder cleanup 
+1. Fixes empty folder cleanup
 
 #### Details:
 - Fixes incorrent path objects on post-runtime folder cleanup

CONTRIBUTING.md

@@ -23,7 +23,7 @@ If your issue appears to be a bug, and hasn't been reported, open a new issue. P
 Providing the following information will allow your issue to be dealt with quickly:
 
 - **Overview of the Issue** - always attach you logs to any new issue.
-- **Version** - what version of Cyberdrop-DL are you running. You should always update to the latest release before opening an issue since 
+- **Version** - what version of Cyberdrop-DL are you running. You should always update to the latest release before opening an issue since
 the issue may have been fixed already
 - **Motivation for or Use Case** - explain what are you trying to do and why the current behavior is a bug for you
 - **Operating System** - what OS and version are you using
@@ -44,10 +44,10 @@ Before you submit your Pull Request (PR) consider the following guidelines:
 - Follow [code style conventions](#code-style)
 - Commit your changes using a descriptive commit message
 - Push your fork to GitHub
-- In GitHub, create a pull request to the `master` branch of the repository. 
-- Add a description to your PR. If the PR is small (such as a typo fix), you can go brief. 
+- In GitHub, create a pull request to the `master` branch of the repository.
+- Add a description to your PR. If the PR is small (such as a typo fix), you can go brief.
 If it contains a lot of changes, it's better to write more details.
-If your changes are user-facing (e.g. a new feature in the UI, a change in behavior, or a bugfix) 
+If your changes are user-facing (e.g. a new feature in the UI, a change in behavior, or a bugfix)
 please include a short message to add to the changelog.
 - Wait for a maintainer to review your PR and then address any comments they might have.
 

cyberdrop_dl/main.py

@@ -15,6 +15,7 @@
 from cyberdrop_dl.managers.manager import Manager
 from cyberdrop_dl.scraper.scraper import ScrapeMapper
 from cyberdrop_dl.ui.ui import program_ui
+from cyberdrop_dl.utils.args.browser_cookie_extraction import get_cookies_from_browser
 from cyberdrop_dl.utils.logger import (
     log,
     log_spacer,
@@ -28,8 +29,6 @@
     send_webhook_message,
     sent_apprise_notifications,
 )
-from cyberdrop_dl.utils.args.browser_cookie_extraction import get_cookies_from_browser
-
 
 
 def startup() -> Manager:
@@ -68,10 +67,13 @@ async def runtime(manager: Manager) -> None:
             manager.task_group = task_group
             await scrape_mapper.start()
 
+
 def pre_runtime(manager):
     """Actions to complete before main runtime"""
     if manager.config_manager.settings_data["Browser_Cookies"]["auto_import"]:
         get_cookies_from_browser(manager)
+
+
 async def post_runtime(manager: Manager) -> None:
     """Actions to complete after main runtime, and before ui shutdown."""
     log_spacer(20)

cyberdrop_dl/ui/prompts/settings_authentication_prompts.py

@@ -149,7 +149,7 @@ def edit_forum_authentication_values_prompt(manager: Manager) -> None:
                 get_cookies_from_browser(manager, "opera_gx")
             elif action == "vivaldi":
                 get_cookies_from_browser(manager, "vivaldi")
-            elif action =="chromium":
+            elif action == "chromium":
                 get_cookies_from_browser(manager, "chromium")
             return
 

cyberdrop_dl/utils/args/browser_cookie_extraction.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from functools import wraps
+from http.cookiejar import MozillaCookieJar
 from typing import TYPE_CHECKING
 
 import browser_cookie3
@@ -9,8 +10,6 @@
 from rich.console import Console
 
 from cyberdrop_dl.utils.dataclasses.supported_domains import SupportedDomains
-from http.cookiejar import MozillaCookieJar
-
 
 if TYPE_CHECKING:
     from collections.abc import Callable
@@ -69,12 +68,14 @@ def wrapper(*args, **kwargs) -> CookieJar:
             inquirer.confirm(message="Press enter to continue").execute()
 
     return wrapper
+
+
 @cookie_wrapper
-def get_cookies_from_browser(manager: Manager, browser: str=None) -> None:
+def get_cookies_from_browser(manager: Manager, browser: str = None) -> None:
     """Get the cookies for the supported sites"""
     manager.path_manager.cookies_dir.mkdir(exist_ok=True)
-    browser=browser or manager.config_manager.settings_data["Browser_Cookies"]["browser"]
-    browser=browser.lower()  if browser else None
+    browser = browser or manager.config_manager.settings_data["Browser_Cookies"]["browser"]
+    browser = browser.lower() if browser else None
     for domain in SupportedDomains.supported_forums:
         cookies = get_cookie(browser, domain)
         cookie_jar = MozillaCookieJar()

cyberdrop_dl/utils/args/config_definitions.py

@@ -126,9 +126,10 @@
         "sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}",
     },
     "Browser_Cookies": {
-        "browser":"Chrome",
+        "browser": "Chrome",
         "auto_import": False,
-}}
+    },
+}
 
 global_settings: dict = {
     "General": {

docs/reference/configuration-options/global-settings.md

@@ -160,7 +160,7 @@ prev downloads are files that are match with the hash and size given, and are no
 
 Current files are files that were either downloaded or a file was skipped for already existing when dedupe_already_downloaded is true
 
-If enabled then at least one existing previous download will be kept on system. 
+If enabled then at least one existing previous download will be kept on system.
 If not enabled all previous downloads will be deleted
 
 </details>

docs/reference/configuration-options/settings.md

@@ -123,7 +123,7 @@ Only grab the first POST/ALBUM from a FILE_HOST_PROFILE
 No FORUM limit, no FORUM_POST limit, no FILE_HOST_PROFILE limit, maximum of 20 items from any FILE_HOST_ALBUM:
 
     --maximum-number-of-children 0 0 0 20
-``` 
+```
 
 </details>
 
@@ -193,7 +193,7 @@ Cyberdrop-DL will output the links it fails to scrape, and the reason in CSV for
 
 * discord\_webhook\_url
 
-The URL of the Discord webhook that you want to send download stats to. You can add the optional tag `attach_logs=` as a prefix to include a copy of the main log as an attachment. 
+The URL of the Discord webhook that you want to send download stats to. You can add the optional tag `attach_logs=` as a prefix to include a copy of the main log as an attachment.
 
 Ex: `attach_logs=https://discord.com/api/webhooks/webhook_id/webhook_token`
 

docs/reference/notifications.md

@@ -14,7 +14,7 @@ To get notifications via discord, you need to provide a discord `webhook_url` in
 
 You can learn how to setup a webhook following the [official discord guide](https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks).
 
-Optionally, you can add the tag `attach_logs=` as a prefix to your webhook url. This will tell CDL to include a copy of the main log as an attachment to Discord. 
+Optionally, you can add the tag `attach_logs=` as a prefix to your webhook url. This will tell CDL to include a copy of the main log as an attachment to Discord.
 
 ## Notifications to other services (via Apprise)
 
@@ -25,7 +25,7 @@ Cyberdrop-DL uses [Apprise](https://github.com/caronc/apprise) to send notificat
 To send notifications via Apprise, you need to create an `apprise.txt` file inside `AppData/Configs/<config_name>`, where `<config_name>` if the config you want to use. The file must contain a list of URLs and they must be in the format of one of the supported apprise services.
 
 You can check the full list of supported services [here](https://github.com/caronc/apprise/wiki) and the URL format than each one uses [here]( https://github.com/caronc/apprise?tab=readme-ov-file#supported-notifications).
- 
+
 Apprise services also support the `attach_logs=` tag to send the main log as an attachment.
 
 ### Troubleshooting Apprise notifications
@@ -35,15 +35,14 @@ Cyberdrop-DL will show you a message at the end of a run telling you if the appr
 
 ## Examples
 
-To get notifications via email, use this URL format in your `apprise.txt` file: 
+To get notifications via email, use this URL format in your `apprise.txt` file:
 
 > mailto://user:password@domain.com
 
-With the tag `attach_logs` it would look like this: 
+With the tag `attach_logs` it would look like this:
 
 > attach_logs=mailto://user:password@domain.com
 
-Using `attach_logs` on the `webhook_url` config option: 
+Using `attach_logs` on the `webhook_url` config option:
 
 > attach_logs=https://discord.com/api/webhooks/webhook_id/webhook_token
-

docs/reference/supported-websites.md

@@ -22,7 +22,7 @@ description: These are the websites supported by Cyberdrop-DL
 | ImgBox                                                                | Album: `/g/...` Images: `/...` Direct Links                                                                                                                                                                                                                                                                                                                                                              |
 | Img.kiwi                                                              | Album: `/album/...` Image: `/image/...` Direct Links                                                                                                                                                                                                                                                                                                                                                     |
 | Imgur                                                                 | Album: `/a/...` Image: `/...` Direct Links                                                                                                                                                                                                                                                                                                                                                               |
-| JPG.Church JPG.Homes JPG.Pet JPEG.Pet JPG1.Su JPG2.Su JPG3.Su JPG4.Su JPG5.Su | Album: `/a/...` Image: `/img/...` Direct Links                                                                             
+| JPG.Church JPG.Homes JPG.Pet JPEG.Pet JPG1.Su JPG2.Su JPG3.Su JPG4.Su JPG5.Su | Album: `/a/...` Image: `/img/...` Direct Links  
 | Imagepond.net | Album: `/a/...` Image: `/img/...` Direct Links                                                                                                                                                                                                                                                                                                                                               |
 | Kemono                                                                | Patreon Model: `/patreon/user/...` FanBox Model: `/fanbox/user/...` Fantia Model: `/fantia/user/...` Afdian Model: `/afdian/user/...` Boosty Model: `/boosty/user/...` DLSite Model: `/dlsite/user/...` Gumroad Model: `/gumroad/user/...` SubscribeStar Model: `/subscribestar/user/...` Discord Server Channel: `/discord/server/...#...` Individual Post: `/<service>/user/.../post/...` Direct Links |
 | MediaFire                                                             | Folder: `/folder/...` File: `/file/...`                                                                                                                                                                                                                                                                                                                                                                  |
@@ -41,12 +41,12 @@ description: These are the websites supported by Cyberdrop-DL
 | Scrolller                                                             | Subreddit: `/r/...`                                                                                                                                                                                                                                                                                                                                                                                      |
 | Toonily                                                               | Webtoon: `/webtoon/...` Chapter: `/webtoon/.../...` Direct Links                                                                                                                                                                                                                                                                                                                                         |
 | Tokyomotion.net                                                       | Video: `/video/...` Photo: `/photo/...` `/user/<user>/favorite/photos` Albums: `/user/<user>/albums/`, `/album/...` Playlist: `/user/<user>/favorite/videos` Profiles: `/user/<user>` Search Results: `/search?...`                                                                                                                                                                                      |
-| XBunkr                                                                | Albums: `/a/...` Direct Links                                                                                                                                                                                                                                                                                                                                                                            |      
+| XBunkr                                                                | Albums: `/a/...` Direct Links                                                                                                                                                                                                                                                                                                                                                                            |  
 | XXXBunker                                                             | Video: `/<video_id>` Search Results: `/search/...`                                                                                                                                                                                                                                                                                                                                                       |
 
 ### Content Hosts - Password Protected<a href="#content-hosts-password-protected" id="content-hosts-password-protected"></a>
 
-Cyberdrop-DL can download password protected files and folders from these hosts. User must include the password as a query parameter in the input URL, adding `?password=<URL_PASSWORD>` to it. 
+Cyberdrop-DL can download password protected files and folders from these hosts. User must include the password as a query parameter in the input URL, adding `?password=<URL_PASSWORD>` to it.
 
 Example: `https://cyberfile.me/folder/xUGg?password=1234`
 

scripts/release/start_linux.sh

@@ -58,4 +58,4 @@ echo
 echo Installing / Updating Cyberdrop-DL
 pip uninstall -y -q -q cyberdrop-dl
 pip install --upgrade "cyberdrop-dl-patched>=5.7,<6.0" && clear && cyberdrop-dl $COMMANDLINE_ARGS
-echo
+echo

scripts/release/start_macOS.command

@@ -58,4 +58,4 @@ echo
 echo Installing / Updating Cyberdrop-DL
 pip uninstall -y -q -q cyberdrop-dl
 pip install --upgrade "cyberdrop-dl-patched>=5.7,<6.0" && clear && cyberdrop-dl $COMMANDLINE_ARGS
-echo
+echo

scripts/release/start_windows.bat

@@ -10,7 +10,7 @@ set "COMMANDLINE_ARGS="
 rem Check the installed Python version
 chcp 65001
 set MIN_PYTHON_VER=3.11
-set MAX_PYTHON_VER=3.13 
+set MAX_PYTHON_VER=3.13
 if not defined PYTHON (set PYTHON=python)
 "%PYTHON%" -c "import sys; MIN_PYTHON_VER = tuple(map(int, '%MIN_PYTHON_VER%'.split('.'))); MAX_PYTHON_VER = tuple(map(int, '%MAX_PYTHON_VER%'.split('.'))); current_version = sys.version_info; exit(0 if (current_version >= MIN_PYTHON_VER and current_version < MAX_PYTHON_VER) else 1)"
 
@@ -47,4 +47,4 @@ echo:
 echo Installing / Updating Cyberdrop-DL
 pip uninstall -y -q -q cyberdrop-dl
 pip install --upgrade "cyberdrop-dl-patched>=5.7,<6.0" && cls && cyberdrop-dl %COMMANDLINE_ARGS%
-pause
+pause