Skip to content

Commit

Permalink
⚡️ optim: 优化脚本
Browse files Browse the repository at this point in the history
  • Loading branch information
henryzhuhr committed Jun 5, 2024
1 parent 628c4b1 commit 82b316d
Show file tree
Hide file tree
Showing 11 changed files with 63 additions and 51 deletions.
22 changes: 14 additions & 8 deletions docs/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ outline: deep

# 模型部署

## 准备模型

如果训练了自定义模型,或者下载了预训练模型,可以跳过这一步

<!--@include: ./download-pretrian.md-->

## 导出模型

提供一个导出脚本 `scripts/train.sh`,复制一份到项目目录下进行自定义修改(推荐)
Expand Down Expand Up @@ -42,13 +48,13 @@ export ENV_PATH=$BASE_ENV_PATH/.env/$ENV_NAME
::: code-group

```shell [使用 venv]
bash scripts/create-python-env.sh -e venv -ni
#zsh scripts/create-python-env.sh -e venv -ni # zsh
bash scripts/create-python-env.sh -e venv
#zsh scripts/create-python-env.sh -e venv # zsh
```

```shell [使用 conda]
bash scripts/create-python-env.sh -e conda -ni
#zsh scripts/create-python-env.sh -e conda -ni # zsh
bash scripts/create-python-env.sh -e conda
#zsh scripts/create-python-env.sh -e conda # zsh
```

:::
Expand Down Expand Up @@ -100,13 +106,13 @@ export ENV_PATH=$BASE_ENV_PATH/.env/$ENV_NAME
::: code-group

```shell [使用 venv]
bash scripts/create-python-env.sh -e venv -ni
#zsh scripts/create-python-env.sh -e venv -ni # zsh
bash scripts/create-python-env.sh -e venv
#zsh scripts/create-python-env.sh -e venv # zsh
```

```shell [使用 conda]
bash scripts/create-python-env.sh -e conda -ni
#zsh scripts/create-python-env.sh -e conda -ni # zsh
bash scripts/create-python-env.sh -e conda
#zsh scripts/create-python-env.sh -e conda # zsh
```

:::
Expand Down
14 changes: 14 additions & 0 deletions docs/download-pretrian.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
下载预训练模型

::: code-group

```shell [bash]
bash scripts/download-yolov5.sh yolov5s # 仅下载单个模型
# bash scripts/download-yolov5.sh # 下载所有模型
```
```shell [zsh]
zsh scripts/download-yolov5.sh yolov5s # 仅下载单个模型
# zsh scripts/download-yolov5.sh # 下载所有模型
```

:::
2 changes: 2 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,5 @@ hero:
text: 项目文档
link: /install
---


15 changes: 8 additions & 7 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ MacOS 系统不支持 CUDA Toolkit,可以使用 CPU 训练模型 (Yolov5 项

## 安装环境

这里安装的环境指的是需要训练的环境,如果不需要训练而是直接部署,请转至 「[模型部署](./deploy)」 文档

提供两种方式安装, venv 或 conda

- **venv** : 如果没有安装,请安装
Expand Down Expand Up @@ -110,24 +112,23 @@ zsh Miniconda3-latest-MacOSX-arm64.sh
```shell
cp scripts/variables.sh scripts/variables.custom.sh
```

- 修改 `export PIP_QUIET=true``false` 可以查看安装过程的详细信息\
- `CACHE_DIR`: 用于存放一些缓存文件,例如 `yolov5/requirements.txt`,默认为项目目录下的 `.cache`
- 安装过程会自动检测 `CUDA_VERSION` 以安装对应的 PyTorch 版本,否则默认安装 CPU 版本的 PyTorch;如果电脑有 NVIDIA GPU 但是不想安装 CUDA Toolkit 到全局系统(需要 sudo)可以取消注释 `export CUDA_VERSION=12.1` 以安装对应的 PyTorch 版本

运行会自动检测是否存在用户自定义的环境变量 `scripts/variables.custom.sh` ,如果存在则使用自定义的环境变量,否则使用默认的环境变量 `scripts/variables.sh`

执行命令自动创建并且激活虚拟环境**可以重复执行该脚本获取激活环境的提示信息或者安装依赖**
执行命令自动创建并且激活虚拟环境,默认使用 `venv`**可以重复执行该脚本获取激活环境的提示信息或者安装依赖**

::: code-group

```shell [使用 venv 创建虚拟环境]
bash scripts/create-python-env.sh -e venv
#zsh scripts/create-python-env.sh -e venv # zsh
bash scripts/create-python-env.sh -i # -i 自动安装依赖
#zsh scripts/create-python-env.sh -i # zsh
```

```shell [使用 conda 创建虚拟环境]
bash scripts/create-python-env.sh -e conda
#zsh scripts/create-python-env.sh -e conda # zsh
bash scripts/create-python-env.sh -e conda -i # -i 自动安装依赖
#zsh scripts/create-python-env.sh -e conda -i # zsh
```

:::
Expand Down
15 changes: 2 additions & 13 deletions docs/train.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,9 @@ outline: deep

## 下载预训练模型

下载预训练模型
<!--@include: ./download-pretrian.md-->

::: code-group

```shell [bash]
bash scripts/download-yolov5.sh yolov5s # 仅下载单个模型
# bash scripts/download-yolov5.sh # 下载所有模型
```
```shell [zsh]
zsh scripts/download-yolov5.sh yolov5s # 仅下载单个模型
# zsh scripts/download-yolov5.sh # 下载所有模型
```

:::
## 训练模型

提供一个训练脚本 `scripts/train.sh`,复制一份到项目目录下进行自定义修改(推荐)

Expand Down
4 changes: 1 addition & 3 deletions scripts/base.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,11 @@ CYAN=$(echo -en '\033[00;36m')

function print_base { echo -e "$1- [$2] $3${DEFAULT}"; }
function print_info { print_base "$CYAN" "INFO" "$1"; }
function print_tip { print_base "$CYAN" "TIP" "$1"; }
function print_tip { print_base "$YELLOW" "TIP" "$1"; }
function print_success { print_base "$GREEN" "SUCCESS" "$1"; }
function print_warning { print_base "$YELLOW" "WARNING" "$1"; }
function print_error { print_base "$RED" "ERROR" "$1"; }


function run_script {
if [ ! -f "$1" ]; then
echo "$1 not found"
Expand All @@ -24,7 +23,6 @@ function run_script {
}



if [ -f "scripts/variables.custom.sh" ]; then
run_script "scripts/variables.custom.sh"
print_success "Loaded custom variables: 'scripts/variables.custom.sh'"
Expand Down
16 changes: 10 additions & 6 deletions scripts/create-python-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ source scripts/base.sh

# Set default values
env_provider="venv"
install=true

install=false
verbose=false

while [ $# -gt 0 ]
do
Expand All @@ -18,10 +18,14 @@ case $key in
shift # shift past argument
shift # shift past value
;;
-ni|--no-install)
install=false
-i|--install)
install=true
shift # shift past argument
;;
-v|--verbose)
verbose=true
shift
;;
-h|--help)
echo "Usage: create-python-env.sh [OPTIONS]"
echo "Options:"
Expand All @@ -38,7 +42,7 @@ case $key in
esac
done

print_info "Environment Provider set to: $env_provider"
print_info "Python environment provider set to: $env_provider"

if [ "$env_provider" = "venv" ]; then
run_script "scripts/utils/python-activate.venv.sh"
Expand All @@ -54,7 +58,7 @@ if [ "$install" = true ]; then
print_info "Installing Python requirements..."
run_script "scripts/utils/python-install-requirements.sh"
else
print_warning "Skipping Python requirements installation..."
print_warning "Skipping Python requirements installation. If you want to install them, add the '-i' or '--install' flag."
fi

print_activate_env_message
Expand Down
7 changes: 3 additions & 4 deletions scripts/utils/python-activate.conda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,16 @@ fi

eval "$(conda shell.$(basename $SHELL) hook)"
conda activate $ENV_PATH
print_success "Activated $(python --version) in ($ENV_PATH)"

print_success "Environment $(python --version) Activated from ($ENV_PATH)"

function print_activate_env_message {
echo ""
print_info "Run command below to activate the environment:"
print_tip "Run command below to activate the environment:"
echo ""
echo "source ~/.`basename $SHELL`rc"
echo "conda activate $ENV_PATH"
echo ""
print_info "Then run command below to deactivate the environment:"
print_tip "Then run command below to deactivate the environment:"
echo ""
echo "source ~/.`basename $SHELL`rc"
echo "conda deactivate"
Expand Down
7 changes: 4 additions & 3 deletions scripts/utils/python-activate.venv.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,21 @@ if [ ! -d $ENV_PATH ]; then
python3 -m venv $ENV_PATH
print_success "Create Python environment in '$ENV_PATH'"
else
print_success "Python environment '$ENV_PATH' already exists."
print_info "Python environment '$ENV_PATH' already exists."
fi

source $ENV_PATH/bin/activate
print_success "Environment $(python --version) Activated from '$ENV_PATH'"

function print_activate_env_message {
echo ""
print_info "Run command below to activate the environment:"
print_tip "Run command below to activate the environment:"
echo ""
echo "source ~/.`basename $SHELL`rc"
echo "source $ENV_PATH/bin/activate"
echo ""

print_info "if you delete the environment, run command below:"
print_tip "if you delete the environment, run command below:"
echo ""
echo "source ~/.`basename $SHELL`rc"
echo "rm -rf $ENV_PATH"
Expand Down
6 changes: 3 additions & 3 deletions scripts/utils/python-install-requirements.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
print_info "Using Python: $(which python3)"
print_info "Installing Python requirements..."

if [ -z "${PIP_QUIET}" ]; then
PIP_QUIET=false
if [ -z "${verbose}" ]; then
verbose=false
fi

if [ "$PIP_QUIET" = true ]; then
if [ "$verbose" = false ]; then
PIP_QUIET_FLAG="-q"
else
PIP_QUIET_FLAG=""
Expand Down
6 changes: 2 additions & 4 deletions scripts/variables.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,17 @@ export ENV_PATH=$BASE_ENV_PATH/.env/$ENV_NAME
# ================== Project Variables ==================
export CACHE_DIR=$PROJECT_HOME/.cache


# ================== Python Variables ==================
# CUSTOM_PYTHON_VERSION=3.12 # -- Uncomment and set to the desired Python version (only for conda)
export PIP_QUIET=true # -- pip install with quiet/verbose

# ================== Enabling CUDA ==================
# ================== Enable CUDA ==================
# -- Variables related to CUDA should be written to ~/.bashrc instead of here
# -- Uncomment to Overwrite the CUDA version if needed
# export CUDA_VERSION=12.1
# export CUDA_HOME="/usr/local/cuda-${CUDA_VERSION}"
# export PATH="$CUDA_HOME/bin:$PATH"

# ================== Enabling OpenVINO ==================
# ================== Enable OpenVINO ==================
export OPENVINO_HOME=/opt/intel/openvino_2024
if [ -d "$OPENVINO_HOME" ]; then
source $OPENVINO_HOME/setupvars.sh
Expand Down

0 comments on commit 82b316d

Please sign in to comment.