Unsupervised Learning of Optical Flow With Non-Occlusion From Geometry

Abstract

Optical flow estimation is a fundamental problem of computer vision and has many applications in the fields of robot learning and autonomous driving. This paper reveals novel geometric principles of optical flow based on the insight and detailed definition of non-occlusion. Then, two novel loss functions are proposed for the unsupervised learning of optical flow based on the geometric principles of non-occlusion. Specifically, after the occlusion part of the images are masked, the flowing process of pixel coordinate points is carefully considered and geometric constraints are conducted based on the geometric principles of optical flow. First, the optical flow vectors of neighboring pixel coordinate points in the first frame will not intersect during the pixel displacement to the second frame. Secondly, when the cluster containing adjacent four pixel coordinate points in the first frame moves to the second frame, no other pixel coordinate points will flow into the quadrilateral formed by them. According to the two geometrical constraints, the optical flow non-intersection loss and the optical flow non-blocking loss in the non-occlusion regions are proposed. Two loss functions punish the irregular and inexact optical flows in the non-occlusion regions. The experiments on datasets demonstrated that the proposed unsupervised losses of optical flow based on the geometric principles in non-occlusion regions make the estimated optical flow more refined in detail, and improve the performance of unsupervised learning of optical flow. In addition, the experiments training on synthetic data and evaluating on real data show that the generalization ability of optical flow network is improved by our proposed unsupervised approach. Our codes are open sourced at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IRMVLab/NccFlow</uri> .