EMO-MVS: Error-Aware Multi-Scale Iterative Variable Optimizer for Efficient Multi-View Stereo

Abstract

Efficient dense reconstruction of objects or scenes has substantial practical implications, which can be applied to different 3D tasks (for example, robotics and autonomous driving). However, because of the expensive hardware required and the overall complexity of the all-around scenarios, efficient dense reconstruction using lightweight multi-view stereo methods has received much attention from researchers. The technological challenge of efficient dense reconstruction is maintaining low memory usage while rapidly and reliably acquiring depth maps. Most of the current efficient multi-view stereo (MVS) methods perform poorly in efficient dense reconstruction, this poor performance is mainly due to weak generalization performance and unrefined object edges in the depth maps. To this end, we propose EMO-MVS, which aims to accomplish multi-view stereo tasks with high efficiency, which means low-memory consumption, high accuracy, and excellent generalization performance. In detail, we first propose an iterative variable optimizer to accurately estimate depth changes. Then, we design a multi-level absorption unit that expands the receptive field, which efficiently generates an initial depth map. In addition, we propose an error-aware enhancement module, enhancing the initial depth map by optimizing the projection error between multiple views. We have conducted extensive experiments on challenging datasets Tanks and Temples and DTU, and also performed a complete visualization comparison on the BlenedMVS validation set (which contains many aerial scene images), achieving promising performance on all datasets. Among the lightweight MVS methods with low-memory consumption and fast inference speed, our F-score on the online Tanks and Temples intermediate benchmark is the highest, which shows that we have the best competitiveness in terms of balancing the performance and computational cost.