Recovering three-dimensional velocity and establishing stereo correspondence from binocular image flows

Abstract

Optical flow fields from parallel stereo cameras are analyzed to determine the 3-D motion of the camera platform with respect to objects in view and to establish stereo correspondence. A linear system of equations, whose coefficients consist of the moments of image velocities and coordinates in the stereo images, is solved to recover the 3-D motion components. This system of equations is derived under the assumption that all feature points have stereo matches (points without stereo matches are called drop-ins and drop-outs); however, point-to-point stereo correspondences are not required. No assumptions about the 3-D structure of the scene are made. Once the 3-D motion components are computed, stereo correspondence for a pair of potentially matching points is established by comparing three computed depth values: Z l obtained from the monocular optical flow equations for the left image; Z r , obtained from the same equations for the right image; and Z s , obtained from stereo disparity. For correct matches these three depths must be nearly equal. The method is applied to numerically generated and laboratory image sequences with and without drop-ins and dropouts. In cases where the percentage of drop-ins and drop-outs is large, an iterative scheme is used to detect and eliminate the drop-ins and drop-outs, thus improving the motion estimates.