Remote Sensing Image Super-resolution Research Articles

Information Purification Network for Remote Sensing Image Super-Resolution

Recently, several well-performing deep convolutional neural networks were proposed for remote sensing image super-resolution (SR). However, these methods rarely consider that remote sensing images are corruptible by additional noise, blurring, and other factors. Therefore, to eliminate the interference of these factors, especially the noise, we propose a novel information purification network (IPN) for remote sensing image SR. The proposed information purification block (IPB) can process channel-wise features differently by channel separation and rescale spatial-wise features adaptively through the proposed multi-scale spatial attention mechanism. We further design an information group to explore a more powerful expressive combination of IPBs. Moreover, long and short skip connections can transmit abundant low-frequency information, making IPBs pay more attention to high-frequency information. We mix the images under various degradation models as training data in the training phase. In this way, the network can directly reconstruct various degraded images. Experiments on AID and UC Merced Land-Use datasets under multiple degradation models demonstrate that the proposed IPN performs better than state-of-the-art methods.

SR-FEINR: Continuous Remote Sensing Image Super-Resolution Using Feature-Enhanced Implicit Neural Representation

Remote sensing images often have limited resolution, which can hinder their effectiveness in various applications. Super-resolution techniques can enhance the resolution of remote sensing images, and arbitrary resolution super-resolution techniques provide additional flexibility in choosing appropriate image resolutions for different tasks. However, for subsequent processing, such as detection and classification, the resolution of the input image may vary greatly for different methods. In this paper, we propose a method for continuous remote sensing image super-resolution using feature-enhanced implicit neural representation (SR-FEINR). Continuous remote sensing image super-resolution means users can scale a low-resolution image into an image with arbitrary resolution. Our algorithm is composed of three main components: a low-resolution image feature extraction module, a positional encoding module, and a feature-enhanced multi-layer perceptron module. We are the first to apply implicit neural representation in a continuous remote sensing image super-resolution task. Through extensive experiments on two popular remote sensing image datasets, we have shown that our SR-FEINR outperforms the state-of-the-art algorithms in terms of accuracy. Our algorithm showed an average improvement of 0.05 dB over the existing method on ×30 across three datasets.

Multi-frame super-resolution of remote sensing images using attention-based GAN models

Multi-frame super-resolution (MFSR) of remote sensing (RS) imageries becomes a critical research topic with the launch of new satellites having video capturing capability and the advancement of artificial intelligence techniques. In this study, an attention-based Generative Adversarial Network (GAN) algorithm is proposed for the multi-frame remote sensing image super-resolution (MRSISR). Firstly, we introduced an attention module to the generator and designed a space-based net that worked on every single frame for better temporal information extraction. Secondly, we proposed a novel attention module for better spatial and spectral information extraction. Thirdly, we applied an attention-based discriminator for the discriminative ability improvement of the discriminator. We implemented several experiments with the state-of-the-art models and the proposed approach using SpaceNet7 and Jilin-1 datasets. We quantitatively and qualitatively compared the results of different multi-frame super-resolution models.

A Closed-Loop Network for Single Infrared Remote Sensing Image Super-Resolution in Real World

Single image super-resolution (SISR) is to reconstruct a high-resolution (HR) image from a corresponding low-resolution (LR) input. It is an effective way to solve the problem that infrared remote sensing images are usually suffering low resolution due to hardware limitations. Most previous learning-based SISR methods just use synthetic HR-LR image pairs (obtained by bicubic kernels) to learn the mapping from LR images to HR images. However, the underlying degradation in the real world is often different from the synthetic method, i.e., the real LR images are obtained through a more complex degradation kernel, which leads to the adaptation problem and poor SR performance. To handle this problem, we propose a novel closed-loop framework that can not only make full use of the learning ability of the channel attention module but also introduce the information of real images as much as possible through a closed-loop structure. Our network includes two independent generative networks for down-sampling and super-resolution, respectively, and they are connected to each other to get more information from real images. We make a comprehensive analysis of the training data, resolution level and imaging spectrum to validate the performance of our network for infrared remote sensing image super-resolution. Experiments on real infrared remote sensing images show that our method achieves superior performance in various training strategies of supervised learning, weakly supervised learning and unsupervised learning. Especially, our peak signal-to-noise ratio (PSNR) is 0.9 dB better than the second-best unsupervised super-resolution model on PROBA-V dataset.

Multi-scale hybrid attention graph convolution neural network for remote sensing images super-resolution

The super-resolution (SR) method of remote sensing image based on deep learning has achieved satisfactory performance. However, when faced with remote sensing images with large size difference and high complexity, the image SR algorithm based deep learning cannot restore more accurate information, resulting in the reconstructed images with blur problem. To solve this problem, we propose a multi-scale hybrid attention graph convolution neural network for remote sensing image SR (MAGSR). Specifically, we construct a chain module that is composed of a multi-scale hybrid attention module and graph convolution neural network to extract more multi-scale deep features and multi-scale high-frequency detail information. Moreover, we design a hybrid attention mechanism, which combines the advantages of spatial attention and channel attention to learn the spatial dependence on each channel, obtaining more abundant critical high-frequency information. Our proposed MAGSR algorithm obtains the highest PSNR and SSIM values on the Google dataset of SIRI-WHU_earth, UCMerced and NWPU-RESISC45. Compared with the suboptimal algorithm DRCAN, our proposed MAGSR achieve 0.03 dB improvements for PSNR. All experimental results show that our proposed MAGSR is superior to other mainstream SR methods and can reconstruct clear SR remote sensing image with better visual effect.

Dual-Path Adversarial Generation Network for Super-Resolution Reconstruction of Remote Sensing Images

Satellite remote sensing images contain adequate ground object information, making them distinguishable from natural images. Due to the constraint hardware capability of the satellite remote sensing imaging system, coupled with the surrounding complex electromagnetic noise, harsh natural environment, and other factors, the quality of the acquired image may not be ideal for follow-up research to make suitable judgment. In order to obtain clearer images, we propose a dual-path adversarial generation network model algorithm that particularly improves the accuracy of the satellite remote sensing image super-resolution. This network involves a dual-path convolution operation in a generator structure, a feature mapping attention mechanism that first extracts important feature information from a low-resolution image, and an enhanced deep convolutional network to extract the deep feature information of the image. The deep feature information and the important feature information are then fused in the reconstruction layer. Furthermore, we also improve the algorithm structure of the loss function and discriminator to achieve a relatively optimal balance between the output image and the discriminator, so as to restore the super-resolution image closer to human perception. Our algorithm was validated on the public UCAS-AOD datasets, and the obtained results showed significantly improved performance compared to other methods, thus exhibiting a real advantage in supporting various image-related field applications such as navigation monitoring.

Learning Dynamic Scale Awareness and Global Implicit Functions for Continuous-Scale Super-Resolution of Remote Sensing Images

The mainstream remote sensing image (RSI) super-resolution (SR) algorithms treat tasks with different scale factors independently, and a single model can only process a fixed integer scale factor. However, in practical applications, it is important to continuously super-resolve RSIs to multiple resolutions, as different resolutions present various levels of detail. Retraining the model for each scale factor consumes huge computational resources and storage space. Existing continuous-scale SR models employ static convolutions, and most are designed for natural scenes, ignoring dynamic feature extraction needs for different scale factors and the inherent properties of RSIs. In addition, efficiently obtaining the continuous representation of RSIs and avoiding the artifacts of RSI SR results is still a challenging problem. To address the above problems, we propose a scale-aware dynamic network (SADN) for RSI continuous-scale SR. First, we devise a scale-aware dynamic convolutional (SAD-Conv) layer to handle the strong randomness of the RSI textural distribution and achieve dynamic feature extraction according to scale factors. Second, we devise a continuous-scale upsampling module (CSUM) with the multi-bilinear global implicit function (MBGIF) for any-scale upsampling. The CSUM constructs multiple feature spaces with asymptotic resolutions to approximate the continuous representation of an image, and then, the MBGIF makes full use of multiresolution features to map arbitrary coordinates to spectral values. We evaluate our SADN using various benchmarks, and the experimental results show that the CSUM can efficiently achieve continuous-scale upsampling while maintaining excellent objective and visual performance. Moreover, our SADN uses fewer parameters and even outperforms the state-of-the-art fixed-scale SR methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hanlinwu/SADN</uri> .

Gradient Prior Dilated Convolution Network for Remote Sensing Image Super-Resolution

Super-resolution (SR) aims to recover highresolution (HR) image from a single or multiple low-resolution (LR) images, compensating for the limitations of satellite sensor imaging. Deep convolutional neural networks (CNN) have made great achievement in remote sensing image super resolution. In this letter, we propose a novel gradient prior dilated convolutional network (GPDCN) for remote sensing images SR, which obtainscontextual spatial connections and alleviates structural distortions. The GPDCN comprise a multi-scale feature extraction network and a feature reconstruction network. The former employs a double-path dilated residual block (DPDRB) with dilation convolution to increase a receptive field, a global selfattention module (GSA) to detect long-range reliance among image patches, and a gradient propagation network (GPN) to extract high-level gradient information. The latter uses the MHOA module to reconstruct the feature by collecting the highorder characteristics of multiple frequency bands. Experiments with the Massachusetts_Roads and 3K VEHICLE_SR datasets demonstrate that the GPDCN outperforms recent techniques concerning both quantitative and qualitative measures.

Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy

Deep learning-based algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR). However, increasing network depth and parameters cause a huge burden of computing and storage. Directly reducing the depth or width of existing models results in a large performance drop. We observe that the SR difficulty of different regions in an RSI varies greatly, and existing methods use the same deep network to process all regions in an image, resulting in a waste of computing resources. In addition, existing SR methods generally predefine integer scale factors and cannot perform stepless SR, i.e., a single model can deal with any potential scale factor. Retraining the model on each scale factor wastes considerable computing resources and model storage space. To address the above problems, we propose a saliency-aware dynamic routing network (SalDRN) for lightweight and stepless SR of RSIs. First, we introduce visual saliency as an indicator of region-level SR difficulty and integrate a lightweight saliency detector into the SalDRN to capture pixel-level visual characteristics. Then, we devise a saliency-aware dynamic routing strategy that employs path selection switches to adaptively select feature extraction paths of appropriate depth according to the SR difficulty of subimage patches. Finally, we propose a novel lightweight stepless upsampling module whose core is an implicit feature function for realizing mapping from low-resolution feature space to high-resolution feature space. Comprehensive experiments verify that the SalDRN can achieve a good tradeoff between performance and complexity. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hanlinwu/SalDRN</uri> .

Multi-Scale Fast Fourier Transform Based Attention Network for Remote-Sensing Image Super-Resolution

Recently, with the rise and progress of convolutional neural networks (CNNs), CNN-based remote sensing image super-resolution (RSSR) methods have gained considerable advancement and showed great power for image reconstruction tasks. However, most of these methods cannot handle well the enormous number of objects with different scales contained in remote sensing images and thus limits super-resolution performance. To address these issues, we propose a Multi-Scale Fast Fourier Transform (FFT) based Attention Network (MSFFTAN) which employs a multi-input U-shape structure as backbone for accurate remote sensing image super-resolution. Specifically, we carefully design an FFT-based residual block consisting of an image domain branch and a Fourier domain branch to extract local details and global structures simultaneously. In addition, a Local-Global Channel Attention Block (LGCAB) is developed to further enhance the reconstruction ability of small targets. Finally, we present a Branch Gated Selective Block (BGSB) to adaptively explore and aggregate features from multiple scales and depths. Extensive experiments on two publicly datasets have demonstrated the superiority of MSFFTAN over the state-of-the-art (SOAT) approaches in aspects of both quantitative metrics and visual quality. The peak signal-to-noise ratio of our network is 1.5 dB higher than the SOAT method on the UCMerced LandUse with downscaling factor 2.

Remote Sensing Image Super-Resolution With Residual Split Attention Mechanism

Remote Sensing Image Super-Resolution With Residual Split Attention Mechanism

Lightweight Remote-Sensing Image Super-Resolution via Re-Parameterized Feature Distillation Network

Recent deep learning based-works have made remarkable progress in Remote-Sensing Image Super-Resolution (RSISR). However, the complicated network architecture as well as a huge amount of parameters increase computational cost, hindering their practical deployment. To alleviate this problem, we propose a novel Re-parameterized Feature Distillation Network (ReFDN) for lightweight and efficient RSISR tasks. Feature distillation, refinement, condensation, and enhancement are efficiently integrated into the re-parameterized feature distillation block named ReFDB for lighter and stronger feature extraction. With the help of elaborate re-parameterized convolution (ReConv) design, we further boost the feature refinement capability without extra inference costs. Additionally, we design an efficient channel and spatial attention module (ECSA) to enhance the important objects and regions of the intermediate features adaptively. Conducted on both commonly used datasets and additional Google Earth data, the experimental results demonstrate our method can achieve a good trade-off between SR performance and network complexity. Our code will be publicly available at https://github.com/DaxingZ/ReFDN.

CLSR: Contrastive Learning for Semi-Supervised Remote Sensing Image Super-Resolution

CLSR: Contrastive Learning for Semi-Supervised Remote Sensing Image Super-Resolution

Double Prior Network for Multidegradation Remote Sensing Image Super-Resolution

Image super-resolution (SR) is widely used in remote sensing because it can effectively increase image details. Neural networks have shown remarkable performance in recent years, benefitting from their end-to-end training. However, remote sensing images contain a variety of degradation factors. Neural networks lack flexibility in dealing with these complex issues compared with reconstruction-based approaches. Traditional neural network methods can not take advantage of prior knowledge and lack interpretability. To develop a flexible, accurate, and interpretable algorithm for remote sensing SR, we proposed an effective SR network called YSRNet. It is performed by unfolding a traditional optimization process into a learnable network. Combining conventional reconstruction-based methods and neural networks can significantly improve the algorithm's performance. Since the gradient features of remote sensing images contain valuable information, the total variation (TV) constraints and the deep prior constraints are introduced into the objective function for image SR. Furthermore, we propose an enhanced version called YSRNet+, which can apply attention weights to different prior terms and channels. Compared with the YSRNet, the YSRNet+ enables networks to focus more on useful prior information and improve the interpretability of networks. Experiments on three remote sensing datasets are performed to evaluate the algorithm's effectiveness. The experimental results demonstrate that the proposed algorithm performs better than some state-of-the-art neural network algorithms, especially in the scenario of the multi-degradation factors.

Hybrid Attention-Based U-Shaped Network for Remote Sensing Image Super-Resolution

Hybrid Attention-Based U-Shaped Network for Remote Sensing Image Super-Resolution

Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion-Based Transformer Network for Remote Sensing Image Super-Resolution

Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion-Based Transformer Network for Remote Sensing Image Super-Resolution

Remote Sensing Image Super-Resolution Adversarial Network Based on Reverse Feature Fusion and Residual Feature Dilation

Remote Sensing Image Super-Resolution Adversarial Network Based on Reverse Feature Fusion and Residual Feature Dilation

Continuous Remote Sensing Image Super-Resolution Based on Context Interaction in Implicit Function Space

Despite its fruitful applications in remote sensing, image super-resolution is troublesome to train and deploy as it handles different resolution magnifications with separate models. Accordingly, we propose a highly-applicable super-resolution framework called FunSR, which settles different magnifications with a unified model by exploiting context interaction within implicit function space. FunSR composes a functional representor, a functional interactor, and a functional parser. Specifically, the representor transforms the low-resolution image from Euclidean space to multi-scale pixel-wise function maps; the interactor enables pixel-wise function expression with global dependencies; and the parser, which is parameterized by the interactor’s output, converts the discrete coordinates with additional attributes to RGB values. Extensive experimental results demonstrate that FunSR reports state-of-the-art performance on both fixed-magnification and continuous-magnification settings, meanwhile, it provides many friendly applications thanks to its unified nature. Our code is available at https://github.com/KyanChen/FunSR.

Unpaired remote sensing image super-resolution with content-preserving weak supervision neural network

Unpaired remote sensing image super-resolution with content-preserving weak supervision neural network

A Review of Image Super-Resolution Approaches Based on Deep Learning and Applications in Remote Sensing

At present, with the advance of satellite image processing technology, remote sensing images are becoming more widely used in real scenes. However, due to the limitations of current remote sensing imaging technology and the influence of the external environment, the resolution of remote sensing images often struggles to meet application requirements. In order to obtain high-resolution remote sensing images, image super-resolution methods are gradually being applied to the recovery and reconstruction of remote sensing images. The use of image super-resolution methods can overcome the current limitations of remote sensing image acquisition systems and acquisition environments, solving the problems of poor-quality remote sensing images, blurred regions of interest, and the requirement for high-efficiency image reconstruction, a research topic that is of significant relevance to image processing. In recent years, there has been tremendous progress made in image super-resolution methods, driven by the continuous development of deep learning algorithms. In this paper, we provide a comprehensive overview and analysis of deep-learning-based image super-resolution methods. Specifically, we first introduce the research background and details of image super-resolution techniques. Second, we present some important works on remote sensing image super-resolution, such as training and testing datasets, image quality and model performance evaluation methods, model design principles, related applications, etc. Finally, we point out some existing problems and future directions in the field of remote sensing image super-resolution.