Indoor scene texturing based on single mobile phone images and 3D model fusion

Abstract

ABSTRACTRealistic texture mapping and coherent up-to-date rendering is one of the most important issues in indoor 3-D modelling. However, existing texturing approaches are usually performed manually during the modelling process, and cannot accommodate changes in indoor environments occurring after the model was created, resulting in out-dated and misleading texture rendering. In this study, a structured learning-based texture mapping method is proposed for automatic mapping a single still photo from a mobile phone onto an already-constructed indoor 3-D model. The up-to-date texture is captured using a smart phone, and the indoor structural layout is extracted by incorporating per-pixel segmentation in the FCN algorithm and the line constraints into a structured learning algorithm. This enables real-time texture mapping according to parts of the model, based on the structural layout. Furthermore, the rough camera pose is estimated by pedestrian dead reckoning (PDR) and map information to determine where to map the texture. The experimental results presented in this paper demonstrate that our approach can achieve accurate fusion of 3-D triangular meshes with 2-D single images, achieving low-cost and automatic indoor texture updating. Based on this fusion approach, users can have a better experience in virtual indoor3-D applications.