Multi-view dense matching supported by triangular meshes

Abstract

We present a new procedure to compute dense 3D point clouds from a sequential set of images. This procedure is considered as a second step of a three-step algorithm for 3D reconstruction from image sequences, whose first step consists of image orientation and the last step is shape reconstruction. We assume that the camera matrices as well as a sparse set of 3D points are available and we strive for obtaining a dense and reliable 3D point cloud. Three novel ideas are presented: (1) for sparse tracking and triangulation, the search space for correspondences is reduced to a line segment by means of known camera matrices and disparity ranges are provided by triangular meshes from the already available points; (2) triangular meshes from extended sets of points are used for dense matching, because these meshes help to reconstruct points in weakly textured areas and present a natural way to obtain subpixel accuracy; (3) two non-local optimization methods, namely, 1D dynamic programming along horizontal lines and semi-global optimization were employed for refinement of local results obtained from an arbitrary number of images. All methods were extensively tested on a benchmark data set and an infrared video sequence. Both visual and quantitative results demonstrate the effectiveness of our algorithm.