Skip to content

Latest commit

 

History

History
172 lines (161 loc) · 7.25 KB

README.md

File metadata and controls

172 lines (161 loc) · 7.25 KB
Raw

FlowNet2

Abstract

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a sub-network specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

Citation

@inproceedings{ilg2017flownet,
  title={Flownet 2.0: Evolution of optical flow estimation with deep networks},
  author={Ilg, Eddy and Mayer, Nikolaus and Saikia, Tonmoy and Keuper, Margret and Dosovitskiy, Alexey and Brox, Thomas},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={2462--2470},
  year={2017}
}

Results and Models

Models Training datasets FlyingChairs Sintel (training) KITTI2012 (training) KITTI2015 (training) Log Config Download
clean final EPE Fl-all EPE
FlowNet2CS FlyingChairs 1.59 - - - - - log Config Model
FlowNet2CS Flying Chairs + FlyingThing3d subset - 1.96 3.69 3.50 28.28% 8.23 log Config Model
FlowNet2CSS FlyingChairs 1.55 - - - - - log Config Model
FlowNet2CSS Flying Chairs + FlyingThing3d subset - 1.85 3.57 3.13 25.76% 7.72 log Config Model
FlowNet2CSS-sd Flying Chairs + FlyingThing3d subset + ChairsSDHom - 1.81 3.69 2.98 25.66% 7.99 log Config Model
FlowNet2 FlyingThing3d subset 1.78 3.31 3.02 25.18% 8.02 log Config Model
Models Training datasets ChairsSDHom Log Config Download
Flownet2sd ChairsSDHom 0.37 log Config Model