Sequential Depth Completion With Confidence Estimation for 3D Model Reconstruction

Abstract

This letter addresses a depth-completion problem for sequential data to reconstruct 3D models of outdoor scenes. While many deep-learning-based approaches have recently achieved promising results, their results are not directly applicable to 3D modeling because of several reasons. First, most results contain a lot of outliers because of irregularly distributed sparse measurements. Second, they ignore temporal coherence in sequential frames and produce temporally inconsistent depths. Therefore, we propose a new method that predicts temporally consistent depths with corresponding confidences from sequential frames. The suggested method can efficiently remove the outliers based on confidence estimates, which accurately represent the true prediction errors. The method also produces temporally consistent depths by integrating the depth information of consecutive frames. In addition, we present a 3D-modeling system that reconstructs a globally consistent 3D model in real-time using the results from the proposed depth completion method. Extensive experiments on synthetic and real-world datasets show that our method outperforms the other state-of-the-art methods in terms of both depth-completion and 3D-modeling accuracies.