Wide baseline stereo matching based on scale invariant feature transformation with hybrid geometric constraints

Abstract

Wide baseline stereo matching is a challenging task because of the presence of significant geometric deformations and illumination changes within the images. Based on the scale invariant feature transformation (SIFT) algorithm, this study proposes a new hybrid matching scheme that uses both the feature-based and the area-based methods to find reliable matches from sparse to dense under different geometric constraints. Firstly, the authors propose a SIFT-based robust weighted least squares matching (LSM) method modelled by a two-dimensional (2D) projective transformation to establish the initial correspondences and their local homographies. In this method, a normalised cross correlation metric modified with an adaptive scale and an orientation of the SIFT features (SIFT-NCC) is proposed to find a good initial alignment for the SIFT-LSM. Secondly, a robust matching propagation using the SIFT-NCC starts from the initial matches under an epipolar geometry and the local homography constraints; geometrical consistency checking is used simultaneously to identify the false matches. Thirdly, they use an improved, feature-based SIFT matching method to find the correspondences from the points that are not coplanar in the 3D space under an epipolar constraint only. A bidirectional selection strategy is used to remove the error matches.