Skip to content

Latest commit

 

History

History
166 lines (154 loc) · 6.6 KB

README.md

File metadata and controls

166 lines (154 loc) · 6.6 KB

SGINet

Pytorch implementation of our paper Semantic-Guided Inpainting Network for Complex UrbanScenes Manipulation In ICPR 2020. Please cite with the following Bibtex code:

@INPROCEEDINGS{9412690,
  author={Ardino, Pierfrancesco and Liu, Yahui and Ricci, Elisa and Lepri, Bruno and de Nadai, Marco},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)}, 
  title={Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation}, 
  year={2021},
  volume={},
  number={},
  pages={9280-9287},
  doi={10.1109/ICPR48806.2021.9412690}}

Please follow the instructions to run the code.

Scripts

1. Installation

  • See the sgi_net.yml configuration file. We provide an user-friendly configuring method via Conda system, and you can create a new Conda environment using the command:
conda env create -f sgi_net.yml
conda activate sgi_net
  • Install cityscapesscripts with pip
cd cityscapesScripts
pip install -e .
  • Install nvidia-apex using
sh install_nvidia_apex.sh

2. Data Preprocessing

Cityscapes

  • Please download the Cityscapes dataset from the official website (registration required). After downloading, please put these files under the ~/datasets/cityscapes/ folder and run the following command in order to generate the correct segmentation maps

    cd cityscapesScripts
CITYSCAPES_DATASET=~/datasets/cityscapes/
python cityscapesscripts/preparation/createTrainIdLabelImgs.py

    If you want to use a different number of labels for the segmentation you can change them in the cityscapesScripts/cityscapesscripts/helpers/labels.py file.

    You should end up with the following structure:

    datasets
├── cityscapes
│   ├── leftImg8bit_sequence
│   │   ├── train
│   │   │   ├── aachen
│   │   │   │   ├── aachen_000003_000019_leftImg8bit.png
│   │   │   │   ├── ...
│   │   ├── val
│   │   │   ├── frankfurt
│   │   │   │   ├── frankfurt_000000_000294_leftImg8bit.png
│   │   │   │   ├── ...
│   ├── gtFine
│   │   ├── train
│   │   │   ├── aachen
│   │   │   │   ├── aachen_000003_000019_gtFine_trainIds.png
│   │   │   │   ├── aachen_000003_000019_gtFine_polygons.json
│   │   │   │   ├── aachen_000003_000019_gtFine_instanceIds.png
│   │   │   │   ├── ...
│   │   ├── val
│   │   │   ├── frankfurt
│   │   │   │   ├── frankfurt_000000_000294_gtFine_trainIds.png
│   │   │   │   ├── frankfurt_000000_000294_gtFine_polygons.json
│   │   │   │   ├── frankfurt_000000_000294_gtFine_instanceIds.png
│   │   │   │   ├── ...

  • Then run the script src/preprocess_city.py in order to prepare the dataset.

    Usage

    The script takes as input three parameters:

    • dataroot: Folder where the Cityscape dataset has been extracted.
    • resize_size: New size of the images (width,height). By default the images will not be resized. Default value: (2048,1024)
    • use_multiprocessing: Run the preprocessing in parallel. By default is disabled

    Example

    cd src
python preprocess_city.py --dataroot ~/datasets/cityscapes/ --resize_size 512,256 --use_multiprocessing

  • Copy the train list and the evaluation list from file_list/cityscapes into the dataroot ~/datasets/cityscapes/

    cp file_list/cityscapes/* ~/dataset/cityscapes/

    You should end up with the following structure:

    datasets
├── cityscapes
│   ├── train_img
│   │   ├── aachen_000003_000019_leftImg8bit.png
│   │   ├── ...
│   ├── val_img
│   │   ├── frankfurt_000000_000294_leftImg8bit.png
│   │   ├── ...
│   ├── train_label
│   │   ├── aachen_000003_000019_gtFine_trainIds.png
│   │   ├── ...
│   ├── val_label
│   │   ├── frankfurt_000000_000294_gtFine_trainIds.png
│   │   ├── ...
│   ├── train_inst
│   │   ├── aachen_000003_000019_gtFine_data.json
│   │   ├── aachen_000003_000019_gtFine_instanceIds.png
│   │   ├── ...
│   ├── val_inst
│   │   ├── frankfurt_000000_000294_gtFine_data.json
│   │   ├── frankfurt_000000_000294_gtFine_instanceIds.png
│   │   ├── ...
│   ├── train.txt
│   ├── val.txt

Indian Driving Dataset

TODO

3. Train

Single-GPU Train

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder 
    cd src
sh script/train_paper_cityscapes_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder 
    cd src
sh script/train_paper_cityscapes.sh

Multi-GPU Train

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder 
    cd src
sh script/train_paper_cityscapes_multigpu_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder 
    cd src
sh script/train_paper_cityscapes_multigpu.sh

The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example

Training with Automatic Mixed Precision (AMP) for faster speed

  • Train a model at 256 x 256 resolution with cropping and Pixel Shuffle in the decoder 
    cd src
sh script/train_paper_cityscapes_fp16_multigpu_pixel_shuffle.sh
  • Train a model at 256 x 256 resolution with cropping and deconvolution in the decoder 
    cd src
sh script/train_paper_cityscapes_fp16_multigpu.sh

The example consider a scenario with a single node and two gpus per node. Please change according to your needs. For more information check the DDP example

4. Test

WORK IN PROGRESS

More Training/Test Details

  • Flags: see options/train_options.py and options/base_options.py for all the training flags; see options/test_options.py and options/base_options.py for all the test flags.

Acknowledgments

This code borrows heavily from pix2pixHD.