PM-MVS: PatchMatch multi-view stereo

Abstract

PatchMatch Stereo is a method for generating a depth map from stereo images by repeating spatial propagation and view propagation. The concept of PatchMatch Stereo can be easily extended to Multi-View Stereo (MVS). In this paper, we present PatchMatch Multi-View Stereo (PM-MVS), which is a highly accurate 3D reconstruction method that can be used in various environments. Three techniques are introduced to PM-MVS: (i) matching score evaluation, (ii) viewpoint selection, and (iii) outlier filtering. The combination of normalized cross-correlation with bilateral weights and geometric consistency between viewpoints is used to improve the estimation accuracy of depth and normal maps at object boundaries and poor-texture regions. For each pixel, viewpoints used for stereo matching are carefully selected in order to improve robustness against disturbances such as occlusion, noise, blur, and distortion. Outliers are removed from reconstructed 3D point clouds by a weighted median filter and consistency-based filters assuming multi-view geometry. Through a set of experiments using public multi-view image datasets, we demonstrate that the proposed method exhibits efficient performance compared with conventional methods.