Remote Sensing Image Super-resolution Research Articles

A dual-path feature reuse multi-scale network for remote sensing image super-resolution

A dual-path feature reuse multi-scale network for remote sensing image super-resolution

Efficient degradation representation learning network for remote sensing image super-resolution

The advancements in convolutional neural networks have led to significant progress in image super-resolution (SR) techniques. Nevertheless, it is crucial to acknowledge that current SR methods operate under the assumption of bicubic downsampling as a degradation factor in low-resolution (LR) images and train models accordingly. However, this approach does not account for the unknown degradation patterns present in real-world scenes. To address this problem, we propose an efficient degradation representation learning network (EDRLN). Specifically, we adopt a contrast learning approach, which enables the model to distinguish and learn various degradation representations in realistic images to obtain critical degradation information. We also introduce streamlined and efficient pixel attention to strengthen the feature extraction capability of the model. In addition, we optimize our model with mutual affine convolution layers instead of ordinary convolution layers to make it more lightweight while minimizing performance loss. Experimental results on remote sensing and benchmark datasets show that our proposed EDRLN exhibits good performance for different degradation scenarios, while the lightweight version minimizes the performance loss as much as possible. The Code will be available at: https://github.com/Leilei11111/EDRLN.

Frequency distribution-aware network based on discrete cosine transformation (DCT) for remote sensing image super resolution.

Single-image super-resolution technology based on deep learning is widely used in remote sensing. The non-local feature reflects the correlation information between different regions. Most neural networks extract various non-local information of images in the spatial domain but ignore the similarity characteristics of frequency distribution, which limits the performance of the algorithm. To solve this problem, we propose a frequency distribution aware network based on discrete cosine transformation for remote sensing image super-resolution. This network first proposes a frequency-aware module. This module can effectively extract the similarity characteristics of the frequency distribution between different regions by rearranging the frequency feature matrix of the image. A global frequency feature fusion module is also proposed. It can extract the non-local information of feature maps at different scales in the frequency domain with little computational cost. The experiments were on two commonly-used remote sensing datasets. The experimental results show that the proposed algorithm can effectively complete image reconstruction and performs better than some advanced super-resolution algorithms. The code is available at https://github.com/Liyszepc/FDANet.

Global sparse attention network for remote sensing image super-resolution

Recently, remote sensing images have become popular in various tasks, including resource exploration. However, limited by hardware conditions and formation processes, the obtained remote sensing images often suffer from low-resolution problems. Unlike the high cost of hardware to acquire high-resolution images, super-resolution software methods are good alternatives for restoring low-resolution images. In addition, remote sensing images have a common nature that similar visual patterns repeatedly appear across distant locations. To fully capture these long-range satellite image contexts, we first introduce the global attention network super-resolution method to reconstruct the images. This network improves the performance but introduces unessential information while significantly increasing the computational effort. To address these problems, we propose an innovative method named the global sparse attention network (GSAN) that integrates both sparsity constraints and global attention. Specifically, our method applies spherical locality sensitive hashing (SLSH) to convert feature elements into hash codes, constructs attention groups based on the hash codes, and computes the attention matrix according to similar elements in the attention group. Our method captures valid and useful global information and reduces the computational effort from quadratic to asymptotically linear regarding the spatial size. Extensive qualitative and quantitative experiments demonstrate that our GSAN has significant competitive advantages in terms of performance and computational cost compared with other state-of-the-art methods.

MMDAN: multiwavelet based multiscale dilated attention network for remote sensing image super-resolution

MMDAN: multiwavelet based multiscale dilated attention network for remote sensing image super-resolution

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

A Remote Sensing Image Super-Resolution Reconstruction Model Combining Multiple Attention Mechanisms.

Remote sensing images are characterized by high complexity, significant scale variations, and abundant details, which present challenges for existing deep learning-based super-resolution reconstruction methods. These algorithms often exhibit limited convolutional receptive fields and thus struggle to establish global contextual information, which can lead to an inadequate utilization of both global and local details and limited generalization capabilities. To address these issues, this study introduces a novel multi-branch residual hybrid attention block (MBRHAB). This innovative approach is part of a proposed super-resolution reconstruction model for remote sensing data, which incorporates various attention mechanisms to enhance performance. First, the model employs window-based multi-head self-attention to model long-range dependencies in images. A multi-branch convolution module (MBCM) is then constructed to enhance the convolutional receptive field for improved representation of global information. Convolutional attention is subsequently combined across channels and spatial dimensions to strengthen associations between different features and areas containing crucial details, thereby augmenting local semantic information. Finally, the model adopts a parallel design to enhance computational efficiency. Generalization performance was assessed using a cross-dataset approach involving two training datasets (NWPU-RESISC45 and PatternNet) and a third test dataset (UCMerced-LandUse). Experimental results confirmed that the proposed method surpassed the existing super-resolution algorithms, including Bicubic interpolation, SRCNN, ESRGAN, Real-ESRGAN, IRN, and DSSR in the metrics of PSNR and SSIM across various magnifications scales.

TDEGAN: A Texture-Detail-Enhanced Dense Generative Adversarial Network for Remote Sensing Image Super-Resolution

Image super-resolution (SR) technology can improve the resolution of images and provide clearer and more reliable remote sensing images of high quality to better serve the subsequent applications. However, when reconstructing high-frequency feature areas of remote sensing images, existing SR reconstruction methods are prone to artifacts that affect visual effects and make it difficult to generate real texture details. In order to address this issue, a texture-detail-enhanced dense generative adversarial network (TDEGAN) for remote sensing image SR is presented. The generator uses multi-level dense connections, residual connections, and Shuffle attention (SA) to improve the feature extraction ability. A PatchGAN-style discrimination network is designed to effectively perform local discrimination and helps the network generate rich, detailed features. To reduce the impact of artifacts, we introduce an artifact loss function, which is combined with the exponential moving average (EMA) technique to distinguish the artifacts generated from the actual texture details through local statistics, which can help the network reduce artifacts and generate more realistic texture details. Experiments show that TDEGAN can better restore the texture details of remote sensing images and achieves certain advantages in terms of evaluation indicators and visualization.

A method of degradation mechanism-based unsupervised remote sensing image super-resolution

Remote sensing image (RSI) super-resolution (SR) is an efficient and low-cost technique to achieve high-resolution and high-quality reconstruction images. The quality of RSI SR reconstruction is affected by the prior information contained in the degradation model. Therefore, studying how to incorporate more RSI degradation prior into the degradation model is crucial. This article presents an approach to design the degradation model by extracting degradation factors from the perspective of remote sensing imaging mechanisms. It includes two aspects: simulating the atmospheric scattering effect through RGB channel weights downsampling and the comprehensive degradation effect of the remote sensing imaging platform through combined blurring. Furthermore, we proposed a high-performance RSI SR network based on degradation mechanism (RSN-DM), which includes a degrader D and a generator G, to employ remote sensing prior fully. We conducted experiments on the UC Merced Land-Use and WPU-RESIS45 datasets, demonstrating that our proposed method is effective. Our method achieves state-of-the-art (SOTA) performance in quantitative evaluation and visual quality. Finally, we apply the proposed degradation model to other networks to further validate the model's effectiveness. Therefore, the degradation model proposed in this paper can enhance the performance of remote sensing image super-resolution techniques in practical applications.

Activated Sparsely Sub-Pixel Transformer for Remote Sensing Image Super-Resolution

Transformers have recently achieved significant breakthroughs in various visual tasks. However, these methods often overlook the optimization of interactions between convolution and transformer blocks. Although the basic attention module strengthens the feature selection ability, it is still weak in generating superior quality output. In order to address this challenge, we propose the integration of sub-pixel space and the application of sparse coding theory in the calculation of self-attention. This approach aims to enhance the network’s generation capability, leading to the development of a sparse-activated sub-pixel transformer network (SSTNet). The experimental results show that compared with several state-of-the-art methods, our proposed network can obtain better generation results, improving the sharpness of object edges and the richness of detail texture information in super-resolution generated images.

Uncertainty-driven mixture convolution and transformer network for remote sensing image super-resolution

Recently, convolutional neural networks (CNNs) and Transformer-based Networks have exhibited remarkable prowess in the realm of remote sensing image super-resolution (RSISR), delivering promising results in the field. Nevertheless, the effective fusion of the inductive bias inherent in CNNs and the long-range modeling capabilities encapsulated within the Transformer architecture remains a relatively uncharted terrain in the context of RSISR endeavors. Accordingly, we propose an uncertainty-driven mixture convolution and transformer network (UMCTN) to earn a performance promotion. Specifically, to acquire multi-scale and hierarchical features, UMCTN adopts a U-shape architecture. Utilizing the dual-view aggregation block (DAB) based residual dual-view aggregation group (RDAG) in both encoder and decoder, we solely introduce a pioneering dense-sparse transformer group (DSTG) into the latent layer. This design effectively eradicates the considerable quadratic complexity inherent in vanilla Transformer structures. Moreover, we introduce a novel uncertainty-driven Loss (UDL) to steer the network’s attention towards pixels exhibiting significant variance. The primary objective is to elevate the reconstruction quality specifically in texture and edge regions. Experimental outcomes on the UCMerced LandUse and AID datasets unequivocally affirm that UMCTN achieves state-of-the-art performance in comparison to presently prevailing methodologies.

A spectral and spatial transformer for hyperspectral remote sensing image super-resolution

ABSTRACT Due to the generally low spatial resolution of hyperspectral images (HSIs), early multispectral images lacked corresponding panchromatic bands, and as a result, fusion methods could not be used to enhance resolution. Many researchers have proposed various image super-resolution methods to address these limitations. However, these methods still suffered from issues, such as inadequate feature representation, lack of spectral feature representation, and high computational cost and inefficiency. To address these challenges, a spectral and spatial transformer (SST) algorithm for hyperspectral remote sensing image super-resolution is introduced. This algorithm uses a spatial transformer structure to extract the spatial features between the image pixels and a spectral transformer structure to extract the spectral features within the image pixels. The integration of these two components is applied to HSI super-resolution. After comparative experiments with currently advanced methods on three publicly available hyperspectral datasets, the results consistently show that our algorithm has better performance in both spectral fidelity and spatial restoration. Furthermore, our proposed algorithm was applied to real-world super-resolution experiments in the region of China's Ruoergai National Park, and subsequently, pixel-based classification was conducted on the super-resolution images, the results indicate that our algorithm could also be applied to future remote sensing interpretation tasks.

Multiscale Residual Dense Network for the Super-Resolution of Remote Sensing Images

Multiscale Residual Dense Network for the Super-Resolution of Remote Sensing Images

A Recurrently Complicated Lightweight Network for Superresolution of Remote Sensing Images

A Recurrently Complicated Lightweight Network for Superresolution of Remote Sensing Images

Progressive Multi-Scale Learning for Remote Sensing Image Super-Resolution with Residual Prior

Progressive Multi-Scale Learning for Remote Sensing Image Super-Resolution with Residual Prior

Hyper-Laplacian Prior for Remote Sensing Image Super-Resolution

Hyper-Laplacian Prior for Remote Sensing Image Super-Resolution

Unsupervised Degradation Aware and Representation for Real-World Remote Sensing Image Super-Resolution

Unsupervised Degradation Aware and Representation for Real-World Remote Sensing Image Super-Resolution

Lightweight Remote Sensing Image Super-Resolution via Background-based Multi-scale Feature Enhancement Network

Lightweight Remote Sensing Image Super-Resolution via Background-based Multi-scale Feature Enhancement Network

TTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution.

Transformer-based method has demonstrated promising performance in image super-resolution tasks, due to its long-range and global aggregation capability. However, the existing Transformer brings two critical challenges for applying it in large-area earth observation scenes: (1) redundant token representation due to most irrelevant tokens; (2) single-scale representation which ignores scale correlation modeling of similar ground observation targets. To this end, this paper proposes to adaptively eliminate the interference of irreverent tokens for a more compact self-attention calculation. Specifically, we devise a Residual Token Selective Group (RTSG) to grasp the most crucial token by dynamically selecting the top- k keys in terms of score ranking for each query. For better feature aggregation, a Multi-scale Feed-forward Layer (MFL) is developed to generate an enriched representation of multi-scale feature mixtures during feed-forward process. Moreover, we also proposed a Global Context Attention (GCA) to fully explore the most informative components, thus introducing more inductive bias to the RTSG for an accurate reconstruction. In particular, multiple cascaded RTSGs form our final Top- k Token Selective Transformer (TTST) to achieve progressive representation. Extensive experiments on simulated and real-world remote sensing datasets demonstrate our TTST could perform favorably against state-of-the-art CNN-based and Transformer-based methods, both qualitatively and quantitatively. In brief, TTST outperforms the state-of-the-art approach (HAT-L) in terms of PSNR by 0.14 dB on average, but only accounts for 47.26% and 46.97% of its computational cost and parameters. The code and pre-trained TTST will be available at https://github.com/XY-boy/TTST for validation.

MSWAGAN: Multispectral Remote Sensing Image Super-Resolution Based on Multiscale Window Attention Transformer

MSWAGAN: Multispectral Remote Sensing Image Super-Resolution Based on Multiscale Window Attention Transformer