3D Virtual Urban Scene Reconstruction From a Single Optical Remote Sensing Image

Abstract

This paper presents a low-cost and efficient method for 3D virtual urban scene reconstruction based on multi-source remote sensing big data and deep learning. By integrating maps, satellite optical images, and digital terrain model (DTM), the proposed method achieves a reasonable reconstructed 3D model for complex urban. The method consists of two independent convolutional neural networks (CNN) to process the land cover and the building height extraction. The proposed method is then tested on a 100 km 2 scene in San Diego, USA, including about 30 000 buildings. The land cover classification achieves an overall accuracy (OA) of 80.4% for eight types of land as defined in NLCD 2011 datasets. Building height estimation achieves an average error at 1.9 meters on NYC open data, the building footprint. Furthermore, the scene reconstruction including the estimation of both land cover and building height can be finished in 10 min on a single NVidia Titan X GPU.