Merge pull request #71 from Guovin/dev

Release: v1.0.5

.gitignore

@@ -1,7 +1 @@
-*.log
-user_*.txt
-user_*.py
-config.py
-demo.txt
-result.txt
-source.json
+*.log

CHANGELOG.md

@@ -1,5 +1,13 @@
 # 更新日志（Changelog）
 
+## v1.0.5
+
+### 2024/4/10
+
+- 移除工作流更新，鉴于有少数人反馈工作流甚至账号被封禁的情况，安全起见，只能暂时移除工作流更新机制，后续将增加其它平台部署方案（Removed workflow updates, in view of the feedback from a few people that their workflows and even accounts have been banned, for safety reasons, the workflow update mechanism can only be temporarily removed, and other platform deployment plans will be added in the future）
+- 新增本地更新，同时移除更新频道个数限制，具体使用方法请见快速上手（Added local updates and removed the limit on the number of channel updates. For specific usage, please see the quick start guide）
+- 适配提供方接口位置变更（Adapted to the change of the provider's interface location）
+
 ## v1.0.4
 
 ### 2024/4/8

Pipfile

@@ -16,6 +16,7 @@ selenium = "*"
 selenium-stealth = "*"
 aiohttp = "*"
 bs4 = "*"
+tqdm = "*"
 
 [requires]
 python_version = "*"

README-EN.md

@@ -9,8 +9,10 @@ Customize channel menus, automatically fetch and update the latest live source i
 - Customize templates to generate the channel categories and order you want
 - Interface validation to filter out invalid interfaces
 - Comprehensive sorting based on response time and resolution
+  <s>
 - Scheduled execution, updates every day at 8:00 am Beijing time
 - The maximum number of update channels is 200
+  </s>
 - Set up key focus channels and configure the number of pages fetched separately
 - Pagination results retrieval (configurable number of pages and interfaces)
 - Ensure update timeliness, configure to retrieve interfaces updated within a recent time range

README.md

@@ -9,8 +9,10 @@
 - 自定义模板，生成您想要的频道分类与频道顺序
 - 接口验效，过滤无效接口
 - 按响应时间、分辨率综合权衡排序
+  <s>
 - 定时执行，北京时间每日 8:00 执行更新一次
 - 更新频道数量上限 200 个
+  </s>
 - 可设置重点关注频道，单独配置获取分页的数量
 - 分页结果获取（可配置页数、接口数量）
 - 保证更新时效性，配置获取最近时间范围内更新的接口

docs/tutorial-EN.md

@@ -72,7 +72,56 @@ Adjust the configuration as needed. Below is the default configuration explanati
 | domain_blacklist       | ["epg.pw"]         | Interface domain blacklist, used to filter out interfaces with low-quality, ad-inclusive domains                   |
 | url_keywords_blacklist | []                 | Interface keyword blacklist, used to filter out interfaces containing specific characters                          |
 
-## Step 4: Enable Auto-update
+## Step 4: Run Updates Locally
+
+### 1. Install Python
+
+Please download and install Python from the official website.
+
+### 2. Run the Update
+
+Run the following commands in the terminal under the project directory:
+
+```python
+pip3 install pipenv
+pipenv install
+pipenv run build
+```
+
+### 3. Update the File to the Repository
+
+After the interface update is completed, upload the user_result.txt to your personal repository to complete the update.
+![Username and Repository Name](./images/rep-info.png 'Username and Repository Name')
+https://mirror.ghproxy.com/raw.githubusercontent.com/your github username/repository name (corresponding to the TV created when forking)/master/user_result.txt
+
+## Step 5: Update the Source Code
+
+Since this project will continue to iterate and improve, if you want to get the latest updates, you can do the following:
+
+### 1. Star
+
+Click on the star button at the homepage of my repository to favorite this project (Your star is the motivation for me to keep updating).
+![Star](./images/star.png 'Star')
+
+### 2. Watch
+
+Follow this project to be notified by email about the latest updates and release logs through releases.
+![Watch-activity](./images/watch-activity.png 'Watch All Activity')
+
+### 3. Sync fork
+
+Return to the homepage of your repository. If there are updates to the project, click on "Sync fork" and then "Update branch" to confirm and update to the latest code.
+![Sync-fork](./images/sync-fork.png 'Sync fork')
+
+<center>
+
+## The following content is deprecated
+
+</center>
+
+<s>
+
+## Step 6: Enable Auto-update
 
 If your template and configuration modifications are correct, you can configure Actions to achieve automatic updates
 
@@ -127,29 +176,12 @@ If you can access this link and it returns the updated interface content, then y
 
 - Note: Except for the first execution of the workflow, which requires you to manually trigger it, subsequent executions (default: daily at 8:00 am Beijing time) will be automatically triggered. If you have modified the template or configuration files and want to execute the update immediately, you can manually trigger (2) Run workflow.
 
-## Step 5: Modify Workflow Update Frequency
+## Step 7: Modify Workflow Update Frequency
 
 ![.github/workflows/main.yml](./images/schedule-cron.png '.github/workflows/main.yml')
 If you want to modify the update frequency (default: daily at 8:00 am Beijing time), you can modify the on:schedule:- cron field.
 
 - #### It is strongly discouraged to make modifications, as there is no difference in the content of the interface in a short period of time. Both too frequent updates and high-consumption running workflows may be judged as resource abuse, leading to the risk of the repository and account being banned.
 - #### Please pay attention to the runtime of your workflow. If you find that the execution time is too long, you need to appropriately reduce the number of channels in the template, modify the number of pages and interfaces in the configuration, in order to meet the compliant operation requirements.
 
-## Step 6: Update the Source Code
-
-Since this project will continue to iterate and improve, if you want to get the latest updates, you can do the following:
-
-### 1. Star
-
-Click on the star button at the homepage of my repository to favorite this project (Your star is the motivation for me to keep updating).
-![Star](./images/star.png 'Star')
-
-### 2. Watch
-
-Follow this project to be notified by email about the latest updates and release logs through releases.
-![Watch-activity](./images/watch-activity.png 'Watch All Activity')
-
-### 3. Sync fork
-
-Return to the homepage of your repository. If there are updates to the project, click on "Sync fork" and then "Update branch" to confirm and update to the latest code.
-![Sync-fork](./images/sync-fork.png 'Sync fork')
+</s>

docs/tutorial.md

@@ -71,7 +71,56 @@
 | domain_blacklist | ["epg.pw"] | 接口域名黑名单，用于过滤低质量含广告类域名的接口 |
 | url_keywords_blacklist | [] | 接口关键字黑名单，用于过滤含特定字符的接口 |
 
-## 步骤四：开启自动更新
+## 步骤四：本地运行更新
+
+### 1. 安装 Python
+
+请至官方下载并安装 Python
+
+### 2. 运行更新
+
+项目目录下终端运行以下命令：
+
+```python
+pip3 install pipenv
+pipenv install
+pipenv run build
+```
+
+### 3. 更新文件至仓库
+
+接口更新完成后，将 user_result.txt 上传至个人仓库，即可完成更新
+![用户名与仓库名称](./images/rep-info.png '用户名与仓库名称')
+https://mirror.ghproxy.com/raw.githubusercontent.com/您的github用户名/仓库名称（对应上述Fork创建时的TV）/master/user_result.txt
+
+## 步骤五：更新源代码
+
+由于本项目将持续迭代优化，如果您想获取最新的更新内容，可进行如下操作
+
+### 1. Star
+
+在我的仓库首页点击收藏该项目（您的 Star 是我持续更新的动力）
+![Star](./images/star.png 'Star')
+
+### 2. Watch
+
+关注该项目，后续更新日志将以 releases 发布，届时您将收到邮件通知
+![Watch-activity](./images/watch-activity.png 'Watch All Activity')
+
+### 3. Sync fork
+
+回到您的仓库首页，如果项目有更新内容，点击 Sync fork，Update branch 确认即可更新最新代码
+![Sync-fork](./images/sync-fork.png 'Sync fork')
+
+<center>
+
+## 以下内容已废弃
+
+</center>
+
+<s>
+
+## 步骤六：开启自动更新
 
 如果您的模板和配置修改没有问题的话，这时就可以配置 Actions 来实现自动更新啦
 
@@ -126,29 +175,12 @@ https://mirror.ghproxy.com/raw.githubusercontent.com/您的github用户名/仓
 
 - 注意：除了首次执行工作流需要您手动触发，后续执行（默认北京时间每日 8:00）将自动触发。如果您修改了模板或配置文件想立刻执行更新，可手动触发（2）中的 Run workflow 即可。
 
-## 步骤五：修改工作流更新频率
+## 步骤七：修改工作流更新频率
 
 ![.github/workflows/main.yml](./images/schedule-cron.png '.github/workflows/main.yml')
 如果您想修改更新频率（默认北京时间每日 8:00），可修改 on:schedule:- cron 字段。
 
 - #### 强烈不建议修改，因为短时间内的接口内容并无差异，过高的更新频率与高耗时运行的工作流都有可能被判定为资源滥用，导致仓库与账户被封禁的风险。
 - #### 请留意您的工作流运行时长，若发现执行时间过长，需要适当删减模板中频道数量、修改配置中的分页数量和接口数量，以达到合规的运行要求。
 
-## 步骤六：更新源代码
-
-由于本项目将持续迭代优化，如果您想获取最新的更新内容，可进行如下操作
-
-### 1. Star
-
-在我的仓库首页点击收藏该项目（您的 Star 是我持续更新的动力）
-![Star](./images/star.png 'Star')
-
-### 2. Watch
-
-关注该项目，后续更新日志将以 releases 发布，届时您将收到邮件通知
-![Watch-activity](./images/watch-activity.png 'Watch All Activity')
-
-### 3. Sync fork
-
-回到您的仓库首页，如果项目有更新内容，点击 Sync fork，Update branch 确认即可更新最新代码
-![Sync-fork](./images/sync-fork.png 'Sync fork')
+</s>

main.py

@@ -23,12 +23,14 @@
 )
 import logging
 import os
+from tqdm import tqdm
 
 logging.basicConfig(
     filename="result_new.log",
     filemode="a",
     format="%(message)s",
     level=logging.INFO,
+    encoding="utf-8",
 )
 
 
@@ -41,6 +43,7 @@ def setup_driver(self):
         options.add_experimental_option("excludeSwitches", ["enable-automation"])
         options.add_experimental_option("useAutomationExtension", False)
         options.add_argument("blink-settings=imagesEnabled=false")
+        options.add_argument("--log-level=3")
         driver = webdriver.Chrome(options=options)
         stealth(
             driver,
@@ -57,15 +60,21 @@ def __init__(self):
         self.driver = self.setup_driver()
 
     async def visitPage(self, channelItems):
+        total_channels = sum(len(channelObj) for _, channelObj in channelItems.items())
+        pbar = tqdm(total=total_channels)
         for cate, channelObj in channelItems.items():
             channelUrls = {}
-            for name in channelObj.keys():
+            channelObjKeys = channelObj.keys()
+            for name in channelObjKeys:
+                pbar.set_description(
+                    f"Processing {name}, {total_channels - pbar.n} channels remaining"
+                )
                 isFavorite = name in config.favorite_list
                 pageNum = (
                     config.favorite_page_num if isFavorite else config.default_page_num
                 )
                 infoList = []
-                for page in range(1, pageNum):
+                for page in range(1, pageNum + 1):
                     try:
                         page_url = f"https://www.foodieguide.com/iptvsearch/?page={page}&s={name}"
                         self.driver.get(page_url)
@@ -81,8 +90,6 @@ async def visitPage(self, channelItems):
                             if tables_div
                             else []
                         )
-                        if not any(result.find("tbody") for result in results):
-                            break
                         for result in results:
                             try:
                                 url, date, resolution = getUrlInfo(result)
@@ -114,17 +121,24 @@ async def visitPage(self, channelItems):
                 except Exception as e:
                     print(f"Error on sorting: {e}")
                     continue
+                finally:
+                    pbar.update()
             updateChannelUrlsTxt(cate, channelUrls)
             # await asyncio.sleep(1)
+        pbar.close()
 
     def main(self):
         asyncio.run(self.visitPage(getChannelItems()))
+        for handler in logging.root.handlers[:]:
+            handler.close()
+            logging.root.removeHandler(handler)
         user_final_file = getattr(config, "final_file", "result.txt")
         user_log_file = (
             "user_result.log" if os.path.exists("user_config.py") else "result.log"
         )
         updateFile(user_final_file, "result_new.txt")
         updateFile(user_log_file, "result_new.log")
+        print(f"Update completed! Please check the {user_final_file} file!")
 
 
 UpdateSource().main()