IMG2DSM: Height Simulation From Single Imagery Using Conditional Generative Adversarial Net

Abstract

This letter proposes a groundbreaking approach in the remote-sensing community to simulating the digital surface model (DSM) from a single optical image. This novel technique uses conditional generative adversarial networks whose architecture is based on an encoder–decoder network with skip connections (generator) and penalizing structures at the scale of image patches (discriminator). The network is trained on scenes where both the DSM and optical data are available to establish an image-to-DSM translation rule. The trained network is then utilized to simulate elevation information on target scenes where no corresponding elevation information exists. The capability of the approach is evaluated both visually (in terms of photographic interpretation) and quantitatively (in terms of reconstruction errors and classification accuracies) on subdecimeter spatial resolution data sets captured over Vaihingen, Potsdam, and Stockholm. The results confirm the promising performance of the proposed framework.