Accurate optical flow estimation using adaptive scale-space and 3D structure tensor

Abstract

Computing optical flow for image sequences is often an essential step to many image processing and computer vision applications. In this paper, a novel, unified optical flow estimation method is developed for simultaneously tackling the aperture problem and multiple motions, and consequently, yielding more accurate optical flow estimation. By integrating Gaussian scale-space with 3D structure tensor, the estimation difficulty encountered in multiple motions resulting from multiple video objects has been handled reasonably well. The obtained normal flow is then treated separately from the real flow, by further applying the least-squares estimation, with the assist of the automatic scale selection mechanism, to produce the estimated real flow. Our proposed automatic scale selection for spatial scale-space is developed from the viewpoint of numerical stability, and the condition number is exploited for adaptively choosing local scales (window sizes). For performance evaluation, we adopted the angular error as the quantitative measurement and used several benchmark image sequences. Experimental results show that the accuracy of our optical flow estimation method is superior to several leading algorithms.