Folding patch correspondence for multiview stereo

Abstract

AbstractIn this article, we propose the novel folding patch model which can replace the traditional patch model utilized in patch‐based multiview stereo (MVS) methods to significantly improve the reconstruction results. The patch model is applied as an approximation of the scene surface differential in the geometric estimation procedure. By minimizing the photometric discrepancy of the projection of the patch model on multiple source images, patch‐based MVS algorithms optimize the position and normal values for the 3D hypothesis of the target pixel. The optimization is based on the assumption that the patch model can fit the target scene surface perfectly. However, when it comes to complex scenes crowded with sharp edges, splintery surfaces, or round surfaces, the patch model is inherently not suitable since even from the microscopic perspective these surfaces are not entirely flat. We construct the folding patch model by folding the traditional patch model from the middle line. By adjusting the folding angle and direction, the folding patch model can fit complex surfaces more flexibly. We apply our folding patch model to the representative open‐source patch based multiview stereo (PMVS) and COLMAP, and validate the effectiveness on ETH3D benchmark and data sets captured in nature. The results demonstrate that utilizing the folding patch model can significantly improve the behavior of PMVS and COLMAP, especially on data sets mainly consist of complex surfaces from plants.