Hybrid homographies and fundamental matrices mixing uncalibrated omnidirectional and conventional cameras

Abstract

In this paper, we present a deep analysis of the hybrid two-view relations combining images acquired with uncalibrated central catadioptric systems and conventional cameras. We consider both, hybrid fundamental matrices and hybrid planar homographies. These matrices contain useful geometric information. We study three different types of matrices, varying in complexity depending on their capacity to deal with a single or multiple types of central catadioptric systems. The first and simplest one is designed to deal with para-catadioptric systems, the second one and more complex considers the combination of a perspective camera and any central catadioptric system. The last one is the complete and generic model which is able to deal with any combination of central catadioptric systems. We show that the generic and most complex model sometimes is not the best option when we deal with real images. Simpler models are not as accurate as the complete model in the ideal case, but they provide a better and more accurate behavior in the presence of noise, being simpler and requiring less correspondences to be computed. Experiments with simulated data and real images are performed. To show the potential of these approaches, we develop two applications. The first is the successful matching between perspective images and hyper-catadioptric images using SIFT descriptors. In the second one, using only the hybrid fundamental matrix and the hybrid planar homography we compute the metric localization of the perspective camera inside the catadioptric view in an indoors environment.