Restoration of Semantic-Based Super-Resolution Aerial Images

Abstract

Currently, technologies for remote sensing image processing are actively developing, including both satellite images and aerial images obtained from video cameras of unmanned aerial vehicles. Often such images have artifacts such as low resolution, blurred image fragments, noise, etc. One way to overcome such limitations is to use modern technologies to restore super-resolution images based on deep learning methods. The specificity of aerial images is the presentation of texture and structural elements in a higher resolution than in satellite images, which objectively contributes to better results of restoration. The article provides a classification of super-resolution methods based on the main architectures of deep neural networks, namely convolutional neural networks, visual transformers and generative adversarial networks. The article proposes a method for reconstructing super-resolution aerial images SemESRGAN taking into account semantic features by using an additional deep network for semantic segmentation during the training stage. The total loss function, including adversarial losses, pixel-level losses, and perception losses (feature similarity), is minimized. Six annotated aerial and satellite image datasets CLCD, DOTA, LEVIR-CD, UAVid, AAD, and AID were used for the experiments. The results of image restoration using the proposed SemESRGAN method were compared with the basic architectures of convolutional neural networks, visual transformers and generative adversarial networks. Comparative results of image restoration were obtained using objective metrics PSNR and SSIM, which made it possible to evaluate the quality of restoration using various deep network models.