Field Of Image Super-resolution Research Articles

Hyperspectral Image Super-Resolution Based on Feature Diversity Extraction

Deep learning is an important research topic in the field of image super-resolution. Problematically, the performance of existing hyperspectral image super-resolution networks is limited by feature learning for hyperspectral images. Nevertheless, the current algorithms exhibit some limitations in extracting diverse features. In this paper, we address limitations to existing hyperspectral image super-resolution networks, focusing on feature learning challenges. We introduce the Channel-Attention-Based Spatial–Spectral Feature Extraction network (CSSFENet) to enhance hyperspectral image feature diversity and optimize network loss functions. Our contributions include: (a) a convolutional neural network super-resolution algorithm incorporating diverse feature extraction to enhance the network’s diversity feature learning by elevating the matrix rank, (b) a three-dimensional (3D) feature extraction convolution module, the Channel-Attention-Based Spatial–Spectral Feature Extraction Module (CSSFEM), to boost the network’s performance in both the spatial and spectral domains, (c) a feature diversity loss function designed based on the image matrix’s singular value to maximize element independence, and (d) a spatial–spectral gradient loss function introduced based on space and spectrum gradient values to enhance the reconstructed image’s spatial–spectral smoothness. In contrast to existing hyperspectral super-resolution algorithms, we used four evaluation indexes, PSNR, mPSNR, SSIM, and SAM, and our method showed superiority during testing with three common hyperspectral datasets.

Landsat-8 to Sentinel-2 Satellite Imagery Super-Resolution-Based Multiscale Dilated Transformer Generative Adversarial Networks

Image super-resolution (SR) techniques can improve the spatial resolution of remote sensing images to provide more feature details and information, which is important for a wide range of remote sensing applications, including land use/cover classification (LUCC). Convolutional neural networks (CNNs) have achieved impressive results in the field of image SR, but the inherent localization of convolution limits the performance of CNN-based SR models. Therefore, we propose a new method, namely, the dilated Transformer generative adversarial network (DTGAN) for the SR of multispectral remote sensing images. DTGAN combines the local focus of CNNs with the global perspective of Transformers to better capture both local and global features in remote sensing images. We introduce dilated convolutions into the self-attention computation of Transformers to control the network’s focus on different scales of image features. This enhancement improves the network’s ability to reconstruct details at various scales in the images. SR imagery provides richer surface information and reduces ambiguity for the LUCC task, thereby enhancing the accuracy of LUCC. Our work comprises two main stages: remote sensing image SR and LUCC. In the SR stage, we conducted comprehensive experiments on Landsat-8 (L8) and Sentinel-2 (S2) remote sensing datasets. The results indicate that DTGAN generates super-resolution (SR) images with minimal computation. Additionally, it outperforms other methods in terms of the spectral angle mapper (SAM) and learned perceptual image patch similarity (LPIPS) metrics, as well as visual quality. In the LUCC stage, DTGAN was used to generate SR images of areas outside the training samples, and then the SR imagery was used in the LUCC task. The results indicated a significant improvement in the accuracy of LUCC based on SR imagery compared to low-resolution (LR) LUCC maps. Specifically, there were enhancements of 0.130 in precision, 0.178 in recall, and 0.157 in the F1-score.

Research on Image Super-Resolution Using Attention Mechanisms based on Super-Resolution Generative Adversarial Network

With the continuous advancement of technology, Super-Resolution Generative Adversarial Networks (SRGAN) have played a significant role in the field of image super-resolution, significantly enhancing the resolution of images. However, while SRGAN excels in generating details, sometimes the restored details do not always meet people's expectations. To further enhance the quality of images and make image details clearer, this paper introduces improvements to the architecture and loss functions of the SRGAN network. Specifically, this research draws inspiration from the architecture of ESRGAN, removing the original Batch Normalization layers and introducing a newly designed Residual Block. Leveraging insights from attention mechanisms, we incorporate three layers of convolutional operations and introduce attention mechanisms into these new Residual Blocks. Furthermore, to simplify the computational complexity of the model, this paper simplifies the original loss functions, consolidating the previous four losses into two. These enhancements result in a significantly improved model in capturing visual elements, making key objects in the images more prominent compared to SRGAN. Detailed experimental results demonstrate that this model, while maintaining the clarity of details, provides higher visual quality. These achievements provide valuable insights and inspiration for further research and applications in the field of image super-resolution.

MLWAN: Multi-Scale Learning Wavelet Attention Module Network for Image Super Resolution

Image super resolution (SR) is an important image processing technique in computer vision to improve the resolution of images and videos. In recent years, deep convolutional neural network (CNN) has made significant progress in the field of image SR; however, the existing CNN-based SR methods cannot fully search for background information in the measurement of feature extraction. In addition, in most cases, different scale factors of image SR are assumed to be different assignments and completed by training different models, which does not meet the actual application requirements. To solve these problems, we propose a multi-scale learning wavelet attention network (MLWAN) model for image SR. Specifically, the proposed model consists of three parts. In the first part, low-level features are extracted from the input image through two convolutional layers, and then a new channel-spatial attention mechanism (CSAM) block is concatenated. In the second part, CNN is used to predict the highest-level low-frequency wavelet coefficients, and the third part uses recursive neural networks (RNN) with different scales to predict the wavelet coefficients of the remaining subbands. In order to further achieve lightweight, an effective channel attention recurrent module (ECARM) is proposed to reduce network parameters. Finally, the inverse discrete wavelet transform (IDWT) is used to reconstruct HR image. Experimental results on public large-scale datasets demonstrate the superiority of the proposed model in terms of quantitative indicators and visual effects.

A Hybrid CNN for Image Denoising

Deep convolutional neural networks (CNNs) with strong learning abilities have been used in the field of image super-resolution. However, some CNNs depends on a single deep network to training an image super-resolution model, which will have poor performance in complex screens. To address this problem, we propose a hybrid denoising CNN (HDCNN). HDCNN is composed of a dilated block (DB), RepVGG block (RVB) and feature refinement block (FB), a single convolution. DB combines a dilated convolution, batch normalization (BN), common convolutions, activation function of ReLU to obtain more context information. RVB uses parallel combination of convolution and BN, ReLU to extract complementary width features. FB is used to obtain more accurate information via refining obtained feature from the RVB. A single convolution collaborates a residual learning operation to construct a clean image. These key components make the HDCNN have good performance in image denoising. Experiment shows that the proposed HDCNN enjoys good denoising effect in public datasets.

DSMA: Reference-Based Image Super-Resolution Method Based on Dual-View Supervised Learning and Multi-Attention Mechanism

Reference image based super-resolution methods (RefSR) have made rapid and remarkable progress in the field of image super-resolution (SR) in recent years by introducing additional high-resolution (HR) images to enhance the recovery of low-resolution (LR) images. The existing RefSR methods can rely on implicit correspondence matching to transfer the HR texture from the reference image (Ref) to compensate for the information loss in the input image. However, the differences between low-resolution input images and high-resolution reference images still affects the effective utilization of Ref images, so it is an important challenge to make full use of the information in Ref images to improve the SR performance. In this paper, we propose an image super-resolution method based on dual-view supervised learning and multi-attention mechanism (DSMA). It enhances the learning of important detail features of Ref images and weakens the interference of noisy information by introducing the multi-attention mechanism, while employing dual-view supervision to motivate the network to learn more accurate feature representations. Quantitative and qualitative experiments on these benchmarks, i.e., CUFED5, Urban100 and Manga109, show that DSMA outperforms the state-of-the-art baselines with significant improvements.

Multiscale Recursive Feedback Network for Image Super-Resolution

Deep learning-based networks have achieved great success in the field of image super-resolution. However, many networks do not fully combine high-level and low-level information, and fuse local and global information. A multiscale recursive feedback network (MSRFN) for image super-resolution is proposed. First, multiscale convolution is integrated into the feedback network to propose multiscale projection units that adaptively capture image features of different scales by driving a multipath information flow. Next, recursive learning is applied to multiscale projection groups composed of up- and down-multiscale projection units to construct a feedback module that exploits high-level information to correct the low-level representation and refines the features in the early layers. Then, global residual learning and local residual feedback were combined to provide more contextual information for the final reconstruction. Experimental results demonstrate that MSRFN can predict more high-frequency details and alleviate the ringing effect and checkerboard artifacts inherently in CNN-based models. Even when the training datasets are relatively small, MSRFN is still superior to most state-of-the-art methods, especially for large scaling factors ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 8$ </tex-math></inline-formula> ).

Construction of super-resolution model of remote sensing image based on deep convolutional neural network

With the continuous improvement of satellite remote sensing technology, using super-resolution image reconstruction technology to reconstruct remote sensing images has important application significance for social development. In the generator model proposed in this paper, the standard convolution layer in the residual network structure is replaced by empty convolution to improve the overall performance of the model while keeping the number of parameters unchanged and the receptive field of convolution at each stage unchanged. By analyzing the advantages of residual network, dense connection network, and cavity convolution in the field of image super resolution, an optimized super-resolution reconstruction model of GAN image with cavity convolution is constructed with dense connection block of cavity residue as a generator component. The cloud computing-based service model is introduced into the image reconstruction system, and the background management module is built through the cloud service system, which is responsible for model training, image transmission, image processing request and database reading. Through experimental analysis, it is proved that the whole automatic data processing from automatic matching data to processing data can be completed, and the performance is better than the traditional service mode, which can produce great economic benefits.

Cascading residual–residual attention generative adversarial network for image super resolution

Image super resolution technology plays an important role in the field of computer vision. With the application of deep learning in the field of image super-resolution, the generative adversarial network is applied to image super-resolution and obtains images with great quality. In this paper, we propose a novel generative adversarial network structure called Cascading Residual–Residual Attention Generative Adversarial Network (CRRAGAN). First, this paper proposes a novel and efficient feature extraction module: Cascading Residual–Residual Block, which can extract multi-scale information and low-level cascade information to high-level information. CRRAGAN directly uses the channel attention module to capture low-resolution image key information and fuse it into the next stage feature. Second, a new loss combination function is proposed, a weighted sum of image loss, adversarial loss, perceptual loss, and charbonnier loss, to make the network training more stable. In the end, we compare our proposed method with 15 previous state-of-the-art methods and discuss the performance of different training datasets. Experimental results demonstrate that our model exhibits improved performance.

Super-resolution for remote sensing images via dual-domain network learning

Super-resolution (SR), which aims at recovering a high-resolution (HR) image from single or sequential low-resolution (LR) images, is a widely used technology in image processing. In the field of image SR, convolutional neural networks (CNNs) have attracted increasing attention because of their high-quality performance. However, most CNN-based methods treat each channel-wise feature equally, which lacks discriminative learning ability across feature channels. Furthermore, many methods neglect to fully use information of each convolutional layer. To resolve these problems, we propose a remote sensing image SR method named dense channel attention network (DCAN). In our DCAN, a sequence of residual dense channel attention blocks (RDCABs) is cascaded with a densely connected structure. In each RDCAB, we make full use of the information from all convolution layers via densely connected convolutional layers. In addition, RDCABs utilize the channel attention mechanism to adaptively recalibrate channel-wise feature responses by explicitly modeling the interdependencies between the channels. In addition, our DCAN can make full use of the hierarchical features by densely connecting each RDCAB. Finally, to further improve the SR performance, the proposed DCAN is learned in both the pixel and wavelet domains, and a fusion layer is used to fuse the outputs of these two domains. Extensive quantitative and qualitative evaluations verify the superiority of our proposed method over several state-of-the-art methods.

Image super-resolution based on multi-grained cascade forests

Although neural networks are most commonly used in the field of image super-resolution (SR), methods based on decision trees are still discussed. These kinds of algorithm need less time to compute than others because of their simple structure but still yield high quality image SR. In this paper, we propose an SR algorithm using the multi-grained cascade forest (SRGCF) method. Our algorithm first uses multi-grained scanning to process the spatial relationships of image features, thus the representational learning ability is improved. During the reconstruction process, the image obtained by cascade forest training is used as the input of the next training, therefore, the image features are continuously emphasized. The training of the cascade forest ends when the evaluation value is optimal. Because the decision tree uses a divide-and-conquer strategy, the SR of an image is improved in an iterative manner simply and quickly. Compared with existing methods, our method not only avoids the tradeoff between reconstruction quality and run time, but also has a good generalization capability. It can be quickly applied to the many cases of image SR.

Image Super-Resolution Based on Sparse Representation via Direction and Edge Dictionaries

Sparse representation has recently attracted enormous interests in the field of image super-resolution. The sparsity-based methods usually train a pair of global dictionaries. However, only a pair of global dictionaries cannot best sparsely represent different kinds of image patches, as it neglects two most important image features: edge and direction. In this paper, we propose to train two novel pairs of Direction and Edge dictionaries for super-resolution. For single-image super-resolution, the training image patches are, respectively, divided into two clusters by two new templates representing direction and edge features. For each cluster, a pair of Direction and Edge dictionaries is learned. Sparse coding is combined with the Direction and Edge dictionaries to realize super-resolution. The above single-image super-resolution can restore the faithful high-frequency details, and the POCS is convenient for incorporating any kind of constraints or priors. Therefore, we combine the two methods to realize multiframe super-resolution. Extensive experiments on image super-resolution are carried out to validate the generality, effectiveness, and robustness of the proposed method. Experimental results demonstrate that our method can recover better edge structure and details.

DESIGN OF DYADIC-INTEGER-COEFFICIENTS BASED BI-ORTHOGONAL WAVELET FILTERS FOR IMAGE SUPER-RESOLUTION USING SUB-PIXEL IMAGE REGISTRATION

This paper presents image super-resolution scheme based on subpixel image registration by the design of a specific class of dyadicinteger-coefficient based wavelet filters derived from the construction of a half-band polynomial. First, the integer-coefficient based halfband polynomial is designed by the splitting approach. Next, this designed half-band polynomial is factorized and assigned specific number of vanishing moments and roots to obtain the dyadic-integer coefficients low-pass analysis and synthesis filters. The possibility of these dyadic-integer coefficients based wavelet filters is explored in the field of image super-resolution using sub-pixel image registration. The two-resolution frames are registered at a specific shift from one another to restore the resolution lost by CCD array of camera. The discrete wavelet transform (DWT) obtained from the designed coefficients is applied on these two low-resolution images to obtain the high resolution image. The developed approach is validated by comparing the quality metrics with existing filter banks.

Kalman filter–based method for image superresolution using a sequence of low-resolution images

The Kalman filter is widely acknowledged as a linear optimal estimator. However, it is usually considered to have taxing computational and storage requirements in the field of image superresolution. This work presents an innovative Kalman filter–based method to solve the inherent problems of the Kalman filter and make it feasible for superresolution applications. Unlike other Kalman filter–based methods, we did not assume the pixels of high-resolution (HR) image are uncorrelated and the covariance matrix is diagonal for the calculation simplification. Instead, we take the spatial correlation of the pixels within a local block of HR image into consideration and propose a block diagonal assumption for the prediction covariance after a new construction way of the column vector which represents the HR image is presented. This significantly reduces the computational complexity during the Kalman filtering. Then, we develop a point-wise Kalman filter method to reduce the computational complexity and decrease the storage requirement. Furthermore, an iterative scheme is proposed to overcome the inaccurate initial estimation problem of superresolution reconstruction. The experiments with synthetic and real data show that the proposed method is effective and outperforms other superresolution methods.