Dense 3D surface reconstruction of large-scale streetscape from vehicle-borne imagery and LiDAR

Abstract

ABSTRACT Accurate and efficient three-dimensional (3D) streetscape reconstruction is the fundamental ability for an exploration vehicle to navigate safely and perform high-level tasks. Recently, remarkable progress has been made in streetscape reconstruction with visual images and light detection and ranging (LiDAR), but they have difficulties either in scaling and reconstructing large-scale outdoors or in efficient processing. To address these issues, this paper proposed an automatic method for incremental dense reconstruction of large-scale 3D streetscapes from coarse to fine at near real time. Firstly, the pose of vehicle is estimated by visual and laser odometry (VLO) and the state-of-the-art pyramid stereo matching network (PSMNet) is introduced to estimate depth information. Then, incremental dense 3D streetscape reconstruction is conducted by key-frame selection and coarse registration with local optimization. Finally, redundant and noise points are removed through multiple filtering, resulting good quality of dense reconstruction. Comprehensive experiments were undertaken to check the visual effect, trajectory pose error and multi-scale model to model cloud comparison (M3C2) based on reference trajectories and reconstructions provided by the state-of-the-art method, showing the precision, recall and F-score of sampling core points (SCPs) are over 80.42%, 71.68% and 77.19%, respectively, which verified the proposed method.