DeepFusion: A simple way to improve traditional multi-view stereo methods using deep learning

Abstract

In this paper, we propose a simple deep learning based method, “DeepFusion”, to improve the results from traditional multi-view stereo (MVS) methods. In our method, we first design a deep convolutional neural network to predict the confidences of the depths produced by the traditional MVS method, which takes the colors, depths, normals and costs as inputs. In particular, we convert the depths into the normalized inverse depths to enhance the accuracy and generality of the network. In addition, we use the plane sweeps in the disparity space to get the top-k matching costs in order to determine the distinctiveness among all costs. Then we propose a novel approach to accurately fuse all depth maps into the final point cloud, which balances the geometric consistency and the predicted confidences. Experimental results on several well-known datasets demonstrate the effectiveness and the generalization ability of the proposed method.