Efficient convex optimization-based texture mapping for large-scale 3D scene reconstruction

Abstract

Texture mapping is a key step in large-scale 3D scene reconstruction, which can greatly enhance visual reality of the reconstructed scenes. However, existing techniques are unable to accomplish this task efficiently due to the high computational complexity of reconstructing large-scale real-world 3D scenes. In this work, we propose a new efficient convex optimization-based approach, i.e. the mesh-based continuous max-flow method, which can be easily implemented and accelerated upon a modern parallel computing platform, e.g. GPU. Particularly, an Markov Random Fields (MRF) based model, i.e. Potts model, is introduced to mathematically formulate the key specific view selection problem; we show that the challenging combinatorial optimization problem can be efficiently solved by resolving its convex relaxation, which recovers textures from images with the proposed duality-based continuous max-flow approach. In addition, visual effects of sharpness and deformation are utilized to define a criterion of evaluating texture quality effectively, and a large 3D triangular mesh is partitioned into structural components so as to reduce memory consumption of the proposed algorithm. The proposed mesh-based continuous max-flow approach for large-scale texture mapping demonstrates its outperformance over state-of-the-art methods, over large-scale public datasets, in both numerical efficiency and visual quality; meanwhile, our GPU-accelerated algorithm can yield 3D textured models with high quality from complex large-scale scenes in minutes.