Semi-global stereo matching under large and spatially variant perceptive deviations

Abstract

Accurate and fast approximate global energy minimisation with semi-global matching (SGM) reconstructs a single visible 3D surface from a stereo pair of images by combining surface cross-sections, or profiles, found in multiple directions using 1D belief propagation (BP). Each energy measures a mismatch between corresponding signals in the images, which is appended with a weighted unevenness of the surface. To account for frequent in practice perceptive (contrast and offset) image deviations, the mismatch is evaluated in terms of mutual information (MI). However, to closely approximate the MI, the deviations have to be spatially uniform and relatively small. Moreover, this conventional two-term energy does not guarantee a unique minimiser; the weight is often hand-picked and may unpredictably affect the minimiser(s), and both the terms cannot be modified independently. Sequential and separate minimisation of these terms partially overcomes the above drawbacks, excludes the weighting, and allows to account for possible spatially variant and large perceptive deviations. For each conjugate pair of epipolar lines in the input images, the first stage employs a symmetric concurrent propagation stereo (SCPS) to build the space of all least-mismatch epipolar profiles of the goal surface. The CP fuses the BP and dynamic programming (DP) energy minimisation and allows for adapting directly to a wide range of perceptive image deviations. The second stage selects the approximately least-uneven surface in the union of the least-mismatch profile-wise spaces using semi-global smoothing (SGS). The latter follows the SGM framework, but to only find the least-uneven cross-sections of the united space in multiple directions and combine them into the final output surface. Experiments with the SCPS-SGS and MI-SGM confirmed efficiency of the proposed two-stage reconstruction under strong perceptive deviations.