A Progressively Enhanced Network for Video Satellite Imagery Superresolution

Abstract

Deep convolutional neural networks (CNNs) have been extensively applied to image or video processing and analysis tasks. For single-image superresolution (SR) processing, previous CNN-based methods have led to significant improvements, when compared to the shallow learning-based methods. However, these CNN-based algorithms with simply direct or skip connections are not suitable for satellite imagery SR because of complex imaging conditions and unknown degradation process. More importantly, they ignore the extraction and utilization of the structural information in satellite images, which is very unfavorable for video satellite imagery SR with such characteristics as small ground targets, weak textures, and over-compression distortion. To this end, this letter proposes a novel progressively enhanced network for satellite image SR called PECNN, which is composed of a pretraining CNN-based network and an enhanced dense connection network. The pretraining part is used to extract the low-level feature maps and reconstructs a basic high-resolution image from the low-resolution input. In particular, we propose a transition unit to obtain the structural information from the base output. Then, the obtained structural information and the extracted low-level feature maps are transmitted to the enhanced network for further extraction to enforce the feature expression. Finally, a residual image with enhanced fine details obtained from the dense connection network is used to enrich the basic image for the ultimate SR output. Experiments on real-world Jilin-1 video satellite images and Kaggle Open Source Dataset show that the proposed PECNN outperforms the state-of-the-art methods both in visual effects and quantitative metrics. Code is available at https://github.com/kuihua/PECNN .