Hybrid stereo matching with a new relaxation scheme of preserving disparity discontinuity

Abstract

A hybrid stereo matching algorithm using a combined edge- and region-based method is proposed to take the advantage of each technique, i.e. an exactly matched point and a full resolution disparity map. Region-based matching is typically more efficient than edge-based matching, however, a region-based matcher lacks the capability of generating an accurate fine resolution disparity map. The generation of such a map can be better accomplished by using edge-based techniques. Accordingly, regions and edges both play important and complimentary roles in a binocular stereo process. Since it is crucial that an efficient and robust stereo system utilizes the most appropriate set of primitives, a nonlinear Laplacian filter is modified to extract proper primitives. Since each pixel value of a second-order differentiated image includes important information for the intensity profile, information such as edge-, signed-, and zero-pixels obtained by the modified nonlinear Laplacian filter, is used to determine the matching strategy. Consequently, the proposed matching algorithm consists of edge-, signed-, and zero- or residual-pixel matching. Different matching strategies are adopted in each matching step. Adaptive windows with variable sizes and shapes are also used to consider the local information of the pixels. In addition, a new relaxation scheme, based on the statistical distribution of matched errors and constraint functions which contain disparity smoothness, uniqueness, and discontinuity preservation, is proposed to efficiently reduce mismatched points in unfavorable conditions. Unlike conventional relaxation schemes, the erosion in an abrupt area of a disparity map is considerably reduced because a discontinuity preservation factor based on a survival possibility function is added to the proposed relaxation. The relaxation scheme can be applied to various methods, such as block-, feature-, region-, object-based matching methods, and so on, by modifying the excitatory set of the smoothness constraint function. Experimental results show that the proposed matching algorithm is effective for various images, even if the image has a high content of noise and repeated patterns. The convergence rate of the relaxation and the output quality are both improved.