SelFlow: Self-Supervised Learning of Optical Flow

The official Tensorflow implementation of SelFlow (CVPR 2019 Oral).

Authors: Pengpeng liu, Michael R. Lyu, Irwin King, Jia Xu

• Testing code and part of pre-trained models are available.
• Training code will come soon. If you need the training code before we release, please refer our previous version DDFlow to implement. DDFlow employs a simple but effective strategy to cope with occluded pixels.

Our SelFlow is the 1st place winner on Sintel Optical Flow Benchmark since November 2018.

Requirements

• Software: The code was developed with python (both python 2 and python 3 are supported), opencv, tensorflow 1.8 and anaconda (optional). It's okay to run without anaconda, but you may need to install the lacking packages by yourself when needed. For tensorflow of different versions, you may need to modify some functions accordingly.

Usage

By default, you can get the testing results using the pre-trained Sintel model by running:

python main.py

Both forward and backward optical flow and their visualization will be written to the output folder.

Please refer to the configuration file template config for a detailed description of the different operating modes.

Testing

• Edit config, set mode = test.
• Create or edit a file, where the first three columns are the input image names, and the last column is the saving name.
• Edit config and set data_list_file to the file directory.
• Edit config and set img_dir to the directory of your image directory.
• Run python main.py.
• Note
  • Supervised pre-trained model: we normalize each channel to be standard normal distribution, please set is_normalize_img=True.
  • Unsupervised pre-trained model: please set is_normalize_img=False.

Training

• Coming soon.

Pre-trained Models

Check models for our pre-trained models on different datasets.

Citation

If you find SelFlow useful in your research, please consider citing:

@inproceedings{Liu:2019:SelFlow, 
title = {SelFlow: Self-Supervised Learning of Optical Flow}, 
author = {Pengpeng Liu and Michael R. Lyu and Irwin King and Jia Xu}, 
booktitle = {CVPR}, 
year = {2019}
}

Acknowledgement

Part of our codes are adapted from PWC-Net and UnFlow, we thank the authors for their contributions.