A CNNs-based method for optical flow estimation with prior constraints and stacked U-Nets

Abstract

Traditional approaches for optical flow estimation always build an energy function which contains data term and smoothness term. However, optimizing the complex function is usually time-consuming. Nowadays, convolution neural networks have been applied in optical flow area. Most of them use large dataset for learning optical flow end-to-end, which can learn motion information from a large amount of prior information prepared in advance. However, these methods rely excessively on the learning ability of the network while ignoring some of well-proven assumptions in traditional approaches. In this paper, inspired by traditional methods, we present a network for learning optical flow, which combines traditional constraints with a supervised network. In the process of network optimization, the brightness constancy, gradient constancy and spatial smoothness assumptions are used to guide the training of network. Moreover, we stack several sub-networks integrated with prior constraints to form a large network for iterative refinement. Our method is tested on several public datasets, such as MPI-Sintel, KITTI2012, KITTI2015, Middlebury. The experimental results show that adding the prior constraints during training can obtain more refined and accurate flow. Compared with other recent methods, our method can achieve state-of-the-art performance on several public benchmarks.