Adaptive large window correlation for optical flow estimation with discrete optimization

Abstract

We propose a scheme for comparing local neighborhoods (window) of image points, to estimate optical flow using discrete optimization. The proposed approach is based on using large correlation windows with adaptive support-weights. We present three new types of weighting constraints derived from image gradient, color statistics and occlusion information. The first type provides gradient structure constraints that favor flow consistency across strong image gradients. The second type imposes perceptual color constraints that reinforce relationship among pixels in a window according to their color statistics. The third type yields occlusion constraints that reject pixels that are seen in one window but not seen in the other. All these constraints contribute to suppress the effect of cluttered background, which is unavoidably included in the large correlation windows. Experimental results demonstrate that each of the proposed constraints appreciably elevates the quality of estimations, and that they jointly yield results that compare favorably to current techniques, especially on object boundaries.