-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
10 changed files
with
140 additions
and
112 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,11 @@ | ||
# Documentation for IMDL-BenCo | ||
|
||
## Introduction | ||
This repo is the documentation for [IMDL-BenCo](https://github.com/scu-zjz/IMDLBenCo) repository, the first comprehensive IMDL benchmark and modular codebase. | ||
This repo is the documentation for [IMDL-BenCo](https://github.com/scu-zjz/IMDLBenCo) repository, the first comprehensive IMDL benchmark and modular codebase. | ||
|
||
|
||
## Local Test for developers | ||
First, you need to install NodeJS, then use this command to host a local server to test the website. | ||
``` | ||
npm run docs:dev | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Framework Design | ||
|
||
## Overview | ||
|
||
The overview of the IMDL-BenCo framework design is shown below: | ||
|
||
![IMDL-BenCo Overview](/images/IMDLBenCo_overview.png) | ||
|
||
The main components include: | ||
- `Dataloader` for data loading and preprocessing | ||
- `Model Zoo` for managing all models and feature extractors | ||
- GPU-accelerated `Evaluator` for calculating evaluation metrics | ||
|
||
These classes are the most carefully designed parts of the framework and can be considered the main contributions of IMDL-BenCo. | ||
|
||
Additionally, there are auxiliary components, including: | ||
- `Data Library` and `Data Manager` for dataset download and management (TODO) | ||
- Global registration mechanism `Register` for mapping `str` to specific `class` or `method`, making it easy to call corresponding models or methods via shell scripts for batch experiments. | ||
- `Visualize tools` for visualization analysis, currently only including Tensorboard. | ||
|
||
And some miscellaneous tools, including: | ||
- `PyTorch optimize tools`, mainly for PyTorch training-related interfaces and tools. | ||
- `Analysis tools`, mainly for various tools used for training or post-training analysis and archiving. | ||
|
||
Each of the aforementioned components is independently designed as a class or function, with appropriate interfaces for interaction between components. Ultimately, they fulfill their respective roles by being imported, called, and combined in various ·、`Training/Testing/Visualizing Scripts`. | ||
|
||
The CLI (Command Line Interface) of the entire IMDL-BenCo framework, similar to `git init` in Git, automatically generates all default `Training/Testing/Visualizing Scripts` scripts in an appropriate working path via `benco init`, for researchers to modify and use subsequently. | ||
|
||
Therefore, users are especially encouraged to modify the content of `Training/Testing/Visualizing Scripts` as needed, making reasonable use of the framework's functions to meet customization needs. According to the ❄️ and 🔥 symbols in the diagram, users are advised to create new classes or modify and design corresponding functions as needed to accomplish specific research tasks. | ||
|
||
Additionally, functions like dataset download and model checkpoint download are also achieved through CLI commands like `benco data`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# 框架设计 | ||
## 概览 | ||
IMDL-BenCo代码框架的设计概览图如下所示: | ||
|
||
![](/images/IMDLBenCo_overview.png) | ||
|
||
主要的组件包含: | ||
- 负责引入数据并进行预处理的`Dataloader` | ||
- 管理全部模型,特征提取器的`Model Zoo` | ||
- 基于GPU加速的`Evaluator`,用于计算评价指标 | ||
|
||
上述类是整个框架中最精心设计的部分,可以认为是IMDL-BenCo的主要贡献。 | ||
|
||
此外还有辅助的一些组件,包含: | ||
- 数据集下载和管理工具`Data Library`和`Data Manager`(TODO) | ||
- 全局的注册机制`Register`,可以实现从`str`到具体`class`或者`method`的映射,便于直接通过shell脚本调用相应的模型或方法,以便批量完成实验。 | ||
- 用于可视化分析结果的`visualize tools`,暂时只包含Tensorboard。 | ||
|
||
以及一些零碎的工具,包含: | ||
- `PyTorch optimize tools`,主要是PyTorch训练相关的接口和工具。 | ||
- `Analysis tools`,主要是各种训练时或训练后,分析存档用的工具。 | ||
|
||
所有上述工具,各自独立地构成了类或者函数,存在交互的组件间留有相应接口。最终,通过在多种`Training/Testing/Visualizing Scrips`中import调用并组合来实现相应的职责。 | ||
|
||
而整个IMDL-BenCo框架的CLI(命令行界面,Command Line Interface)则以类似Git中`git init`的行为,通过`benco init`自动地在合适的工作路径生成所有**默认**的`Training/Testing/Visualizing Scrips`脚本,供研究人员进行后续修改使用。 | ||
|
||
所以,我们尤其鼓励使用者按照需求修改`Training/Testing/Visualizing Scrips`的内容,完成对于框架功能的合理取用,满足自定义的需求。并根据图中的❄️、🔥标志建议、酌情对于标注为🔥的类按需创建新的类或修改、设计相应功能完成最相应的科研任务。 | ||
|
||
此外,数据集下载,模型checkpoint下载等等功能也是通过`benco data`等等CLI指令实现。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# Documentation for IMDL-BenCo | ||
|
||
## Introduction | ||
This repo is the documentation for [IMDL-BenCo](https://github.com/scu-zjz/IMDLBenCo) repository, the first comprehensive IMDL benchmark and modular codebase. | ||
|
||
|
||
## Local Test for developers | ||
First, you need to install NodeJS, then use this command to host a local server to test the website. | ||
``` | ||
npm run docs:dev | ||
``` | ||
Then a website server will run at http://localhost:8080/IMDLBenCo-doc/ | ||
|
||
After changes to the document, the content on the webpage will change automatically. |