Integration of stereo vision and optical flow by using an energy-minimization approach

Abstract

A cooperative method is proposed in which image intensity (brightness) and optical-flow information are integrated into a single stereo technique by modeling the input data as coupled Markov random fields (MRF’s). The Bayesian probabilistic estimation method and the MRF–Gibbs equivalence theory are used to integrate the optical flow and the gray-level intensity information to obtain an energy function that will explicitly represent the depth discontinuity and occlusion constraints on the solution. This energy function involves the similarity in intensity (or edge orientation) and the optical flow between corresponding sites of the left and right images as well as the smoothness constraint on the disparity solution. If a simple MRF is used to model the data, the energy function will yield a poor disparity by smoothing across object boundaries, particularly when occluding objects are present. We use optical-flow information to indicate object boundaries (depth discontinuities) and occluded regions, in order to improve the disparity solution in occluded regions. A stochastic relaxation algorithm (simulated annealing) is used to find a favorable disparity solution by minimization of the energy equation.