Skip to content

Latest commit

 

History

History
138 lines (118 loc) · 11 KB

getting_started.md

File metadata and controls

138 lines (118 loc) · 11 KB
Raw

Getting Started 🛫

Create environment

  • Create conda environment with following command conda create -y -n inspyrenet python
  • Activate environment with following command conda activate inspyrenet
  • Install latest PyTorch from Official Website
    • Linux [2022.11.09]
    pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
  • Install requirements with following command pip install -r requirements.txt

Preparation

  • For training, you may need training datasets and ImageNet pre-trained checkpoints for the backbone. For testing (inference), you may need test datasets (sample images).
  • Training datasets are expected to be located under Train.Dataset.root. Likewise, testing datasets should be under Test.Dataset.root.
  • Each dataset folder should contain images folder and masks folder for images and ground truth masks respectively.
  • You may use multiple training datasets by listing dataset folders for Train.Dataset.sets, such as [DUTS-TR] -> [DUTS-TR, HRSOD-TR, UHRSD-TR].

Backbone Checkpoints

Item Destination Folder OneDrive GDrive
Res2Net50 checkpoint data/backbone_ckpt/*.pth Link Link
SwinB checkpoint data/backbone_ckpt/*.pth Link Link
  • We changed Res2Net50 checkpoint to resolve an error while training with DDP. Please refer to issue #9.

Train Datasets

Item Destination Folder OneDrive GDrive
DUTS-TR data/Train_Dataset/... Link Link

Extra Train Datasets (High Resolution, Optional)

Item Destination Folder OneDrive GDrive
HRSOD-TR data/Train_Dataset/... Link N/A
UHRSD-TR data/Train_Dataset/... Link N/A
DIS-TR data/Train_Dataset/... Link N/A

Test Datasets

Item Destination Folder OneDrive GDrive
DUTS-TE data/Test_Dataset/... Link Link
DUT-OMRON data/Test_Dataset/... Link Link
ECSSD data/Test_Dataset/... Link Link
HKU-IS data/Test_Dataset/... Link Link
PASCAL-S data/Test_Dataset/... Link Link
DAVIS-S data/Test_Dataset/... Link Link
HRSOD-TE data/Test_Dataset/... Link Link
UHRSD-TE data/Test_Dataset/... Link Link

Extra Test Datasets (Optional)

Item Destination Folder OneDrive GDrive
FSS-1000 data/Test_Dataset/... Link N/A
MSRA-10K data/Test_Dataset/... Link N/A
DIS-VD data/Test_Dataset/... Link Link
DIS-TE1 data/Test_Dataset/... Link Link
DIS-TE2 data/Test_Dataset/... Link Link
DIS-TE3 data/Test_Dataset/... Link Link
DIS-TE4 data/Test_Dataset/... Link Link

Train & Evaluate

  • Train InSPyReNet
# Single GPU
python run/Train.py --config configs/InSPyReNet_SwinB.yaml --verbose

# Multi GPUs with DDP (e.g., 4 GPUs)
torchrun --standalone --nproc_per_node=4 run/Train.py --config configs/InSPyReNet_SwinB.yaml --verbose

# Multi GPUs with DDP with designated devices (e.g., 2 GPUs - 0 and 1)
CUDA_VISIBLE_DEVICES=0,1 torchrun --standalone --nproc_per_node=2 run/Train.py --config configs/InSPyReNet_SwinB.yaml --verbose

  • --config [CONFIG_FILE], -c [CONFIG_FILE]: config file path for training.

  • --resume, -r: use this argument to resume from last checkpoint.

  • --verbose, -v: use this argument to output progress info.

  • --debug, -d: use this argument to save debug images every epoch.

  • Train with extra training datasets can be done by just changing Train.Dataset.sets in the yaml config file, which is just simply adding more directories (e.g., HRSOD-TR, HRSOD-TR, UHRSD-TR, ...):

Train:
  Dataset:
      type: "RGB_Dataset"
      root: "data/RGB_Dataset/Train_Dataset"
      sets: ['DUTS-TR'] --> ['DUTS-TR', 'HRSOD-TR', 'UHRSD-TR']
  • Inference for test benchmarks
python run/Test.py --config configs/InSPyReNet_SwinB.yaml --verbose
  • Evaluate metrics
python run/Eval.py --config configs/InSPyReNet_SwinB.yaml --verbose
  • All-in-One command (Train, Test, Eval in single command)
# Single GPU
python Expr.py --config configs/InSPyReNet_SwinB.yaml --verbose

# Multi GPUs with DDP (e.g., 4 GPUs)
torchrun --standalone --nproc_per_node=4 Expr.py --config configs/InSPyReNet_SwinB.yaml --verbose

# Multi GPUs with DDP with designated devices (e.g., 2 GPUs - 0 and 1)
CUDA_VISIBLE_DEVICES=0,1 torchrun --standalone --nproc_per_node=2 Expr.py --config configs/InSPyReNet_SwinB.yaml --verbose

Inference on your own data

  • You can inference your own single image or images (.jpg, .jpeg, and .png are supported), single video or videos (.mp4, .mov, and .avi are supported), and webcam input (ubuntu and macos are tested so far).
python run/Inference.py --config configs/InSPyReNet_SwinB.yaml --source [SOURCE] --dest [DEST] --type [TYPE] --gpu --jit --verbose
  • --source [SOURCE]: Specify your data in this argument.
    • Single image - image.png
    • Folder containing images - path/to/img/folder
    • Single video - video.mp4
    • Folder containing videos - path/to/vid/folder
    • Webcam input: 0 (may vary depends on your device.)
  • --dest [DEST] (optional): Specify your destination folder. If not specified, it will be saved in results folder.
  • --type [TYPE]: Choose between map green, rgba, blur, overlay, and another image file.
    • map will output saliency map only.
    • green will change the background with green screen.
    • rgba will generate RGBA output regarding saliency score as an alpha map. Note that this will not work for video and webcam input.
    • blur will blur the background.
    • overlay will cover the salient object with translucent green color, and highlight the edges.
    • Another image file (e.g., backgroud.png) will be used as a background, and the object will be overlapped on it.
    Examples of TYPE argument

  • --gpu: Use this argument if you want to use GPU.
  • --jit: Slightly improves inference speed when used.
  • --verbose: Use when you want to visualize progress.