A Filtering-Based Framework for Optical Flow Estimation

Abstract

We present a novel optical flow estimation framework that reinterprets most popular algorithms (e.g., the Horn-Schunck model, PDE-based models, TV-based models, and nonlocal-based models) from an iterative filtering perspective. Different regularizers in the related optical flow estimation algorithms can be achieved by adopting different filtering operations. The key merit of the proposed framework is that a variety of filtering models and corresponding optimization strategies, which are used in image processing, can be directly utilized in the regularization term, leading to a convenient way of designing appropriate optical flow estimation algorithms. Under the proposed filtering framework, we demonstrate easily designing optical flow estimation algorithms with respect to translation consistency, rotation consistency, and divergence consistency constraints by adopting different spatial filters, respectively. We also derive a novel optical flow estimation algorithm with 3D filtering-based model for the regularization term, which makes use of nonlocal self-similarity and sparse characteristic of optical flow field. Benefited from the advantages of patch-based nonlocal sparse regularizer, the derived algorithm can remove outliers while preserving sharp flow edges and important motion details. At present, the proposed algorithm achieves a high ranking on the challenging MPI-Sintel dataset and shows good performance on the Kitti flow 2015 and Middlebury datasets.