Large Displacement Optical Flow Estimation Based on Robust Interpolation of Sparse Correspondences

Abstract

Recently, the interpolation of correspondences method has been widely used in optical flow estimation, because it produces an accurate flow field and costs little runtimes. However, most of the existing matching-based optical flow methods are usually susceptible to non-rigid motion and large displacements. We propose in this article a large displacement optical flow estimation method based on robust interpolation of sparse correspondences, named Riscflow. First, we utilize the deep matching model to achieve an initial matching result of two consecutive frames, and then we exploit a grid-based motion statistics optimization scheme to remove the outliers from the initial matching field. Second, we propose a random forest-based motion boundary extraction model and construct a sparse-to-dense interpolation method by using the boundary information to prevent the dense matching field from edge-blurring. Third, we design a global optical flow estimation method by using an energy function to optimize the dense matching field. Finally, we respectively run the proposed method on the MPI-Sintel and UCF101 databases to conduct a comprehensive comparison with some state-of-the-art optical flow approaches including the variational methods, the matching-based methods, and the deep learning-based methods. The comparison results demonstrate that the proposed method has high accuracy and good robustness of optical flow estimation, and especially gains the benefit of edge-preserving under non-rigid motion and large displacements.