Soft-Aligned Gradient-Chaining Network for Height Estimation From Single Aerial Images

Abstract

Extracting elevation information from single aerial images is an important and challenging task in photogrammetry and remote sensing. The difficulty lies in the ambiguity and ill-condition of the 2-D-3-D projection. As neural networks show advantages in solving ill-posed problems, fully convolutional neural networks (FCNs) have been applied to this task and have achieved worthwhile results. However, the previous works simply focus on the application of the neural network structures originally applied to monocular depth estimations or semantic segmentations to single-image elevation estimations. This procedure ignores the task-specific weak correlation problem between the elevation systems of training labels and outputs and the one between heights of adjacent pixels. To address these problems, we use the FCN to generate height differences between adjacent pixels in specific orientations (i.e., oriented elevation gradients), instead of directly generating the elevation values. Subsequently, the elevation gradients in the two orients of XY are combined to form a huge relationship chain, and the relative elevation of each pixel is calculated using the maximum likelihood method. Finally, a new surface-based soft alignment method is used to establish the correlation between the elevation systems of training labels and predicted outputs robustly. The quantitative evaluation on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam data set reveals a significant improvement of the bare FCN. The visual contrast evaluation indicates that the soft-aligned gradient-chaining network can retain more surface features, and its outputs are more reasonable and visually pleasing than the bare FCN.