Sat2Map: Reconstructing 3D Building Roof from 2D Satellite Images

Abstract

Three-dimensional (3D) urban models have gained interest because of their applications in many use cases, such as disaster management, energy management, and solar potential analysis. However, generating these 3D representations of buildings require lidar data, which is usually expensive to collect. Consequently, the lidar data are not frequently updated and are not widely available for many regions in the US. As such, 3D models based on these lidar data are either outdated or limited to those locations where the data is available. In contrast, satellite images are freely available and frequently updated. We propose sat2Map , a novel deep learning-based approach that predicts building roof geometries and heights directly from a single 2D satellite image. Our method first uses sat2pc to predict the point cloud by integrating two distinct loss functions, Chamfer Distance and Earth Mover’s Distance, resulting in a 3D point cloud output that balances overall structure and finer details. Additionally, we introduce sat2height , a height estimation model that estimates the height of the predicted point cloud to generate the final 3D building structure for a given location. We extensively evaluate our model on a building roof dataset and conduct ablation studies to analyze its performance. Our results demonstrate that sat2Map consistently outperforms existing baseline methods by at least 18.6%. Furthermore, we show that our refinement module significantly improves the overall performance, yielding more accurate and fine-grained 3D outputs. Our sat2height model demonstrates a high accuracy in predicting height parameters with a low error rate. Furthermore, our evaluation results show that we can estimate building heights with a median mean absolute error of less than 30 cm while still preserving the overall structure of the building.