Self-supervised multi-body scene flow estimation

Abstract

In this paper, we address the problem of scene flow estimation from consecutive stereo pairs. In contrast to the state-of-the-art supervised learning based methods, we propose a self-supervised learning based pipeline that removes the requirement of large-scale ground truth annotations. Specifically, we employ a shared encoder StereoFlowNet to simultaneously learn optical flow estimation and disparity estimation, which not only achieves a compact network representation but also exploits the inherent connections between optical flow estimation and disparity estimation. To leverage the scene structure and motion representations, we propose to utilize a piece-wise planar model based disparity computation and multiple rigid body motion representation of the dynamic scene. In this way, the geometric and motion constraints play strong regularizations for the underlying problem. Experimental results on benchmarking dataset show that our proposed method achieves state-of-the-art performance in both optical flow and disparity estimation.