A trifocal sensor system for 3D reconstruction of architectural models

Abstract

This article describes a fully automated video sensor system for the 3D reconstruction of architectural models from image sequences. The hybrid system unifies triangulation methods of three high-resolution cameras (spatial stereo) with feature-tracking methods over time (temporal stereo). Because standard tracking techniques suffer from outliers between subsequent images, we improve the outlier detection using temporal as well as spatial trifocal information. Furthermore, these spatio-temporal constraints allow the system to perform a guided matching that increases the number of tracked features and is robust to partial occlusions. The fully automatic and reliable calculation of the camera path from those tie points is still a challenging task. The use of multiple calibrated cameras that are fixed on a rig leads to additional constraints, which significantly stabilize the pose estimation process. To achieve a dense surface reconstruction, we propose an efficient spatial image-matching algorithm, which is based on trifocal image rectification and semi-global optimization using mutual information. Our improvements include a symmetric and hierarchical matching strategy with sub-pixel accuracy.