IMG2HEIGHT: height estimation from single remote sensing image using a deep convolutional encoder-decoder network

Abstract

ABSTRACT Height estimation from single remote sensing image is a challenging inherently ambiguous and technically ill-posed problem that we address in this study by resorting to deep learning approach. A spatial enhanced and multi-scale aggregated encoder-decoder network, SM-EDNet, is proposed, which takes a single image as input and produces an estimated height map as output. First, residual network (ResNet) is applied to extract low-level and deep features to cope with the heterogeneous characteristics of remote sensing scenes. Then, the multi-scale context information is aggregated through DenseASPP (Dense Atrous Spatial Pyramid Pooling) by extracting features from multiple dilated convolution layers. The skip connection is constructed by using the structure preserving model, DULR, to aggregate ResNet low-level features and multi-scale high-level features. The deformable convolution module is constructed to enhance the sensitivity to differences in geometric shapes of ground objects. For model training, three-layer deep supervision mechanism is designed to counteract the adverse effects of unstable gradients changes. Experimental results on three benchmark datasets, including ISPRS Vaihingen, ISPRS Potsdam, and DFC2018, show that the proposed method achieves the most outstanding performance compared with the state-of-the-art networks. The source codes are available at: https://github.com/xjh0929/2HEIGHT.