Merge pull request #173 from Ljzd-PRO/devel

Bump to v0.9.0
Ljzd-PRO · Oct 26, 2024 · 492d446 · 492d446
2 parents 0183910 + 0a74df5
commit 492d446
Show file tree

Hide file tree

Showing 28 changed files with 1,651 additions and 681 deletions.
diff --git a/prod.env → .env b/prod.env → .env
diff --git a/.github/actions/pyinstaller/action.yml b/.github/actions/pyinstaller/action.yml
@@ -20,5 +20,5 @@ runs:
       shell: bash
 
     - name: Copy Extra Files
-      run: cp README.md README_zh-CN.md example.env prod.env LICENSE dist
+      run: cp README.md README_zh-CN.md example.env .env LICENSE dist
       shell: bash
diff --git a/.github/actions/setup-python/action.yml b/.github/actions/setup-python/action.yml
@@ -20,5 +20,5 @@ runs:
         architecture: "x64"
         cache: "poetry"
 
-    - run: poetry install --with test,docs,pyinstaller
+    - run: poetry install --with test,docs,pyinstaller --extras urwid
       shell: bash
diff --git a/.gitignore b/.gitignore
@@ -120,7 +120,7 @@ celerybeat.pid
 *.sage.py
 
 # Environments
-.env
+# .env
 .venv
 env/
 venv/

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,28 +2,24 @@
 
 ### 💡 Feature
 
-- Show the download job status each 30s (waiting, running, completed%)
-- Shortened the log length
-  - E.g. `2024-10-05 20:12:37 | WARNING  | ktoolbox.job.runner - Download file already exists, skipping ...`
+- Added a **graphical configuration editor**, making configuration editing simple and convenient.
+  - Run `ktoolbox config-editor`
+- Added a command to generate a complete sample configuration file.
+  - Run `ktoolbox example-env`
+- **Python** versions below 3.8.1 (**<3.8.1**) are no longer supported
 
-### 🪲 Fix
-
-- Fix error when attempting to download files which posses too long names (invalid names) (#150)
-  - For example the wrong filename like this: `https://www.patreon.com/media-u/Z0FBQUFBQm........=#12345678_` \
-    KToolBox can get the correct filename: `6edd5bdae......0e7f913.png`
+[//]: # (### 🪲 Fix)
 
 - - -
 
 ### 💡 新特性
 
-- 每隔 30 秒显示下载任务状态（等待中、运行中、已完成%）
-- 缩短了日志长度
-  - 例如 `2024-10-05 20:12:37 | WARNING  | ktoolbox.job.runner - Download file already exists, skipping ...`
-
-### 🪲 修复
+- 新增 **图形化配置编辑器**，配置编辑将变得简单方便
+  - 运行 `ktoolbox config-editor`
+- 新增命令可生成完整的配置文件样例
+  - 运行 `ktoolbox example-env`
+- **Python** 3.8.1 以下 (**<3.8.1**) 的版本不再受支持
 
-- 修复下载过长文件名（非法文件名）的文件时报错的问题 (#150)
-  - 例如这样的错误文件名：`https://www.patreon.com/media-u/Z0FBQUFBQm........=#12345678_` \
-    KToolBox 可以获取到正确的文件名：`6edd5bdae......0e7f913.png`
+[//]: # (### 🪲 修复)
 
-**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.8.0...v0.9.0
+**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.9.0...v0.10.0
diff --git a/README.md b/README.md
@@ -130,6 +130,9 @@ Goto [Shortcuts for iOS](https://ktoolbox.readthedocs.io/latest/shortcut/) page
 
 Goto [Configuration-Guide](https://ktoolbox.readthedocs.io/latest/configuration/guide/) page for more details.
 
+![KToolBox Configuration Editor](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-2.png)
+![KToolBox Configuration Editor](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-3.png)
+
 ## Other Branches
 
 - Pure Python branch: [🔗pure-py](https://github.com/Ljzd-PRO/KToolBox/tree/pure-py)

diff --git a/README_zh-CN.md b/README_zh-CN.md
@@ -130,6 +130,9 @@ ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --start-time=2024-1-1 -
 
 前往 [配置-向导](https://ktoolbox.readthedocs.io/latest/zh/configuration/guide/) 页面查看更多详情。
 
+![KToolBox 配置编辑器](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-2.png)
+![KToolBox 配置编辑器](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-3.png)
+
 ## 其他分支
 
 - 纯 Python 分支：[🔗pure-py](https://github.com/Ljzd-PRO/KToolBox/tree/pure-py)

diff --git a/SECURITY.md b/SECURITY.md
@@ -8,8 +8,7 @@
 | 3.11.x  | :white_check_mark: |
 | 3.10.x  | :white_check_mark: |
 | 3.9.x   | :white_check_mark: |
-| 3.8.x   | :white_check_mark: |
-| <= 3.7  | :x:                |
+| < 3.8.1 | :x:                |
 
 ## Reporting a Vulnerability
 

diff --git a/docs/en/commands/guide.md b/docs/en/commands/guide.md
@@ -18,6 +18,22 @@ ktoolbox -h
 ktoolbox download-post -h
 ```
 
+## Launch the graphical configuration editor
+
+`config-editor`
+
+```bash
+ktoolbox config-editor
+```
+
+## Generate an example configuration file (`.env`/`prod.env`)
+
+`example-env`
+
+```bash
+ktoolbox example-env
+```
+
 ## Download a specific post
 
 `download-post`

diff --git a/docs/en/configuration/guide.md b/docs/en/configuration/guide.md
@@ -1,6 +1,25 @@
 # Guide
 
-- KToolBox read **`.env` or `prod.env` file** in the work folder or **environment variables** to store configuration
+!!! tip "Graphical Configuration Editor"
+
+    === "Usage"
+
+        Run `ktoolbox config-editor` to launch, 
+        it will make configuration editing simple and convenient.
+
+        - Press `<Esc>` for back, `<Enter>` for select
+        - You can also use the GUI with the mouse
+
+    === "Screenshot"
+
+        ![KToolBox Configuration Editor](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-2.png){ loading=lazy }
+        ![KToolBox Configuration Editor](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-3.png){ loading=lazy }
+
+!!! tip "Generate Example `.env` File"
+
+    Run `ktoolbox example-env` to generate a complete sample configuration file.
+
+- KToolBox load **`.env` or `prod.env` file** in the **working directory** or **environment variables** to store configuration
 - Check [Reference](reference.md) for all configuration options
 - Use `__` to specify the sub option, like `KTOOLBOX_API__SCHEME` means `api.scheme`
 - All configuration options are optional

diff --git a/docs/en/faq.md b/docs/en/faq.md
@@ -45,4 +45,71 @@ KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
 
 # Set the post directory name to its release/publish date and ID, e.g. `[2024-1-1]11223344`
 KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{id}
-```
+```
+
+## How to Configure a Proxy?
+
+You can set the `HTTPS_PROXY`, `HTTP_PROXY`, and `ALL_PROXY` environment variables to achieve this.
+
+Refer to: [HTTPX - Environment Variables](https://www.python-httpx.org/environment_variables/#http_proxy-https_proxy-all_proxy)
+
+For example, set it like this:
+
+```shell
+# Unix Shell
+export HTTPS_PROXY=http://127.0.0.1:7897
+export HTTP_PROXY=http://127.0.0.1:7897
+export ALL_PROXY=socks5://127.0.0.1:7897
+```
+
+```powershell
+# Windows PowerShell
+$env:HTTP_PROXY="http://127.0.0.1:7897"; $env:HTTPS_PROXY="http://127.0.0.1:7897"
+```
+
+## GUI Configuration Editor Cannot Be Opened
+
+!!! warning "Note"
+    [`ktoolbox-pure-py`](https://pypi.org/project/ktoolbox-pure-py/) does not support the graphical configuration editor.
+
+By default, the dependencies for the graphical configuration editor are not installed. You can install them using the following command:
+
+```shell
+pip3 install ktoolbox[urwid]
+```
+
+If you are using pipx:
+
+```shell
+pipx install ktoolbox[urwid] --force
+```
+
+## Kemono API Call Failed
+
+For example:
+
+```
+ktoolbox sync-creator "https://coomer.su/onlyfans/user/hollyharper11" --start-time="2020-05-01" --end-time="2025-01-01"
+
+2024-05-12 12:52:51.477 | INFO     | ktoolbox.cli:sync_creator:271 - Got creator information - {'name': 'hollyharper11', 'id': 'hollyharper11'}
+2024-05-12 12:52:51.479 | INFO     | ktoolbox.action.job:create_job_from_creator:148 - Start fetching posts from creator hollyharper11
+2024-05-12 12:52:56.477 | ERROR    | ktoolbox.api.base:_retry_error_callback:37 - Kemono API call failed - {'ret': APIRet(code=1002, message="1 validation error for Response\n  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\\n<html>\\...>\\n  </body>\\n</html>\\n', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid", exception=1 validation error for Response
+  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\n<html>\...>\n  </body>\n</html>\n', input_type=str]
+    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid, data=None)}
+1 validation error for Response
+  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\n<html>\...>\n  </body>\n</html>\n', input_type=str]
+    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid
+```
+
+This is generally caused by frequent requests, so you can try setting a higher number of API retry attempts.
+
+```dotenv
+# .env / prod.env
+KTOOLBOX_API__RETRY_TIMES=10
+```
+
+You can also set this through the graphical configuration editor: API - retry_times.
+
+## Others
+
+A community-shared usage guide: [#141](https://github.com/Ljzd-PRO/KToolBox/issues/141)
diff --git a/docs/en/index.md b/docs/en/index.md
@@ -37,8 +37,6 @@
   </a>
 </p>
 
-<img src="https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-1.png" alt="Preview">
-
 ## Features
 
 - Support for **multi-thread** downloads (technically, coroutine)

diff --git a/docs/zh/commands/guide.md b/docs/zh/commands/guide.md
@@ -18,6 +18,22 @@ ktoolbox -h
 ktoolbox download-post -h
 ```
 
+## 启动图形化配置编辑器
+
+`config-editor`
+
+```bash
+ktoolbox config-editor
+```
+
+## 生成一个配置文件样例 (`.env`/`prod.env`)
+
+`example-env`
+
+```bash
+ktoolbox example-env
+```
+
 ## 下载指定的作品
 
 `download-post`

diff --git a/docs/zh/configuration/guide.md b/docs/zh/configuration/guide.md
@@ -1,6 +1,24 @@
 # 向导
 
-- KToolBox 读取工作目录下的 **`.env` 或 `prod.env` 文件** 或 **环境变量** 来设定配置
+!!! tip "图形化配置编辑器"
+
+    === "用法"
+
+        运行 `ktoolbox config-editor` 来启动，这会使配置编辑变得简单方便。
+
+        - 按下 `<Esc>` 来返回，按 `<Enter>` 来选择 
+        - 你也可以通过鼠标使用这个 GUI
+
+    === "截图"
+
+        ![KToolBox 配置编辑器](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-2.png){ loading=lazy }
+        ![KToolBox 配置编辑器](https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-3.png){ loading=lazy }
+
+!!! tip "生成示例 `.env` 文件"
+
+    运行 `ktoolbox example-env` 来生成完整的配置文件样例。
+
+- KToolBox 读取 **工作目录** 下的 **`.env` 或 `prod.env` 文件** 或 **环境变量** 来设定配置
 - 前往 [参考](./reference.md) 查看所有配置选项
 - 用 `__` 来指定子选项, 例如 `KTOOLBOX_API__SCHEME` 相当于 `api.scheme`
 - 所有配置选项都是可选的

diff --git a/docs/zh/faq.md b/docs/zh/faq.md
@@ -42,4 +42,71 @@ KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
 
 # 设置作品目录名为其发布日期和ID，例如 `[2024-1-1]11223344`
 KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{id}
-```
+```
+
+## 如何配置代理？
+
+可以通过设置 `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY` 环境变量实现
+
+参考：[HTTPX - Environment Variables](https://www.python-httpx.org/environment_variables/#http_proxy-https_proxy-all_proxy)
+
+例如这样设置：
+
+```shell
+# Unix Shell
+export HTTPS_PROXY=http://127.0.0.1:7897
+export HTTP_PROXY=http://127.0.0.1:7897
+export ALL_PROXY=socks5://127.0.0.1:7897
+```
+
+```powershell
+# Windows PowerShell
+$env:HTTP_PROXY="http://127.0.0.1:7897"; $env:HTTPS_PROXY="http://127.0.0.1:7897"
+```
+
+## 图形化配置编辑器无法打开
+
+!!! warning "注意"
+    [`ktoolbox-pure-py`](https://pypi.org/project/ktoolbox-pure-py/) 不支持图形化配置编辑器
+
+默认情况下，图形化配置编辑器的相关依赖不会被安装，可使用以下命令附带安装：
+
+```shell
+pip3 install ktoolbox[urwid]
+```
+
+如果你用的是 pipx：
+
+```shell
+pipx install ktoolbox[urwid] --force
+```
+
+## Kemono API 调用失败
+
+例如：
+
+```
+ktoolbox sync-creator "https://coomer.su/onlyfans/user/hollyharper11" --start-time="2020-05-01" --end-time="2025-01-01"
+
+2024-05-12 12:52:51.477 | INFO     | ktoolbox.cli:sync_creator:271 - Got creator information - {'name': 'hollyharper11', 'id': 'hollyharper11'}
+2024-05-12 12:52:51.479 | INFO     | ktoolbox.action.job:create_job_from_creator:148 - Start fetching posts from creator hollyharper11
+2024-05-12 12:52:56.477 | ERROR    | ktoolbox.api.base:_retry_error_callback:37 - Kemono API call failed - {'ret': APIRet(code=1002, message="1 validation error for Response\n  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\\n<html>\\...>\\n  </body>\\n</html>\\n', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid", exception=1 validation error for Response
+  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\n<html>\...>\n  </body>\n</html>\n', input_type=str]
+    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid, data=None)}
+1 validation error for Response
+  Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='<!DOCTYPE html>\n<html>\...>\n  </body>\n</html>\n', input_type=str]
+    For further information visit https://errors.pydantic.dev/2.7/v/json_invalid
+```
+
+一般可能是因为请求频繁导致，你可以尝试设置更多的 API 重试次数
+
+```dotenv
+# .env / prod.env
+KTOOLBOX_API__RETRY_TIMES=10
+```
+
+你也可以通过图形化配置编辑器设置：API - retry_times
+
+## 其他
+
+一个社区分享的使用向导：[#141](https://github.com/Ljzd-PRO/KToolBox/issues/141)
diff --git a/docs/zh/index.md b/docs/zh/index.md
@@ -38,8 +38,6 @@
   </a>
 </p>
 
-<img src="https://cdn.jsdelivr.net/gh/Ljzd-PRO/KToolBox@latest/static/preview-1.png" alt="Preview">
-
 ## 功能
 
 - 支持 **多线程** 下载（技术上是协程）

diff --git a/example.env b/example.env
@@ -1,3 +1,8 @@
+##############################################################################
+#  It is recommended to use the graphical configuration editor for editing.  #
+#  Run `ktoolbox config-editor` to launch it.                                #
+##############################################################################
+
 # Download 10 files at the same time.
 KTOOLBOX_JOB__COUNT=10
 
@@ -13,14 +18,7 @@ KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
 # For example: `{title}_{}` > `HelloWorld_b4b41de2-8736-480d-b5c3-ebf0d917561b`, etc.
 # You can also use it with `sequential_filename`. For instance,
 # `[{published}]_{}` > `[2024-1-1]_1.png`, `[2024-1-1]_2.png`, etc.
-KTOOLBOX_JOB__FILENAME_FORMAT=[{published}]_{}
+KTOOLBOX_JOB__FILENAME_FORMAT=[{published}]_{title}_{id}_{}
 
 # Prefix the post directory name with its release/publish date, e.g. `[2024-1-1]HelloWorld`
 KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{title}
-
-# Allocate 102400 Bytes as buffer for each download job
-KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
-
-# Disable SSL certificate verification for Kemono API server and download server
-# It's useful when certificate on Kemono server expired. (SSL: CERTIFICATE_VERIFY_FAILED)
-KTOOLBOX_SSL_VERIFY=False
-Original file line number
+Diff line change
@@ Expand Up / @@ -120,7 +120,7 @@ celerybeat.pid @@
     *.sage.py
     # Environments
-    .env
+    # .env
     .venv
     env/
     venv/
@@ Expand Down @@