Single Image Super-resolution Reconstruction Research Articles

Dual residual and large receptive field network for lightweight image super-resolution

In recent years, there has been rapid progress in single-image super-resolution (SISR) reconstruction technology based on deep learning, but many methods face challenges in practical application due to their excessive computational requirements. To address this issue, numerous lightweight super-resolution (SR) methods have been proposed. Convolutional neural networks (CNNs) have demonstrated outstanding performance in lightweight SR reconstruction tasks. However, existing CNN-based lightweight SR networks still suffer from redundancy due to the use of small convolutional kernels and deeper network architectures for feature extraction. To solve these problems, we propose a novel network called Dual Residual and Large Receptive Field Network (DRLN), which includes two main designs. Firstly, a Large Receptive Field Feature Extraction Block (LRFEB) is proposed, which concatenates large kernel depth separable convolutions and large kernel depth separable dilated convolutions, effectively capturing spatial contextual information without increasing the number of parameters or computational burden, and enhancing the overall performance of the model. Secondly, in the Efficient Feature Distillation Block (EFDB), we innovatively propose a dual residual structure, which greatly improve the learning ability and feature expression richness of the network, and significantly improve the reconstruction performance. Extensive experiments demonstrate that DRLN surpasses BSRN which is the champion of the model complexity track of the NTIRE 2022 Efficient SR Challenge on all benchmark datasets, with fewer parameters and less computation.

Super-resolution reconstruction of single image for latent features

AbstractSingle-image super-resolution (SISR) typically focuses on restoring various degraded low-resolution (LR) images to a single high-resolution (HR) image. However, during SISR tasks, it is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features. This challenge can lead to issues such as model collapse, lack of rich details and texture features in the reconstructed HR images, and excessive time consumption for model sampling. To address these problems, this paper proposes a Latent Feature-oriented Diffusion Probability Model (LDDPM). First, we designed a conditional encoder capable of effectively encoding LR images, reducing the solution space for model image reconstruction and thereby improving the quality of the reconstructed images. We then employed a normalized flow and multimodal adversarial training, learning from complex multimodal distributions, to model the denoising distribution. Doing so boosts the generative modeling capabilities within a minimal number of sampling steps. Experimental comparisons of our proposed model with existing SISR methods on mainstream datasets demonstrate that our model reconstructs more realistic HR images and achieves better performance on multiple evaluation metrics, providing a fresh perspective for tackling SISR tasks.

A review of single image super-resolution reconstruction based on deep learning

A review of single image super-resolution reconstruction based on deep learning

MFFN: Multi‐path feedback fusion network for lightweight image super resolution

AbstractRecently, convolutional neural network (CNN) has shown great power in single image super resolution (SISR) reconstruction, and achieving significant improvements over traditional methods. Despite the great success of these CNN‐based methods, direct application of these methods to some edge devices is impractical due to the large computational overhead required. To address this problem, a novel, lightweight SISR network focusing on speed and accuracy, called the multi‐path feedback fusion network (MFFN), has been designed in this paper. Specifically, in order to extract features more effectively, the authors propose a novel fusion attention feedback block (FAFB) as the main building block of MFFN. The FAFB consists of a backbone branch and several hierarchical branches. The backbone branch is composed of stacked enhanced pixel attentional blocks (EPAB), which are responsible for incremental deep feature learning on the feature map. And the hierarchical branches are responsible for extracting feature maps with different sizes of receptive fields and fusing these feature maps with the features extracted from the trunk branches to achieve multi‐scale feature learning, which the authors refer to this design as the multi‐scale fusion block (MSFB). Extra enhancement information (EIE) is added to each EPAB input, which enables the backbone branch to learn more effectively. On the other hand, the outputs of the cascade branches are further complemented by an additional feedback fusion enhancement block (FFEB) before being fused with the output of the trunk branches to achieve more comprehensive and accurate feature learning. Numerous experiments have shown that MFFN achieves higher accuracy than other state‐of‐the‐art methods on benchmark test sets.

Improved wavelet prediction superresolution reconstruction based on U‐Net

AbstractDeep learning can be used to achieve single‐image superresolution (SR) reconstruction. To address problems encountered during this process, such as the number of network parameters, high training requirements on equipment performance, and inability to downsample certain SR images accurately, an image SR reconstruction algorithm based on deep residual network optimization is proposed. The model introduces wavelet transforms based on the original U‐Net, where the U‐Net is trained to obtain SR wavelet feature images at multiple scales simultaneously. This approach reduces the mapping space for the network to learn low‐ to high‐resolution image mapping, which in turn reduces the training difficulty of the model. In terms of network details, the inverse wavelet transform is used in image upsampling to enhance the sparsity of the reconstruction layer in the original network. The network structure of the U‐Net upsampling is adjusted slightly to enable the network to distinguish wavelet images from feature images, thereby improving the richness of the features extracted by the model. The experimental results show that the peak signal‐to‐noise ratio (PSNR) of the fourfold SR model is 32.35 and 28.68 on the Set5 and Set14 validation sets, respectively. Compared with networks that use wavelet prediction mechanisms, such as the deep wavelet prediction SR (DWSR) and deep wavelet prediction‐based residual SR (DWRSR) models, the PSNR for all the tested public datasets is improved by 0.5. The method yields superior results in terms of both visual effect and PSNR, demonstrating the feasibility of the wavelet prediction mechanism in SR reconstruction and thus offering application value and research significance.

DVDR-SRGAN: Differential Value Dense Residual Super-Resolution Generative Adversarial Network

In the field of single-image super-resolution reconstruction, GAN can obtain the image texture more in line with the human eye. However, during the reconstruction process, it is easy to generate artifacts, false textures, and large deviations in details between the reconstructed image and the Ground Truth. In order to further improve the visual quality, we study the feature correlation between adjacent layers and propose a differential value dense residual network to solve this problem. We first use the deconvolution layer to enlarge the features, then extract the features through the convolution layer, and finally make a difference between the features before being magnified and the features after being extracted so that the difference can better reflect the areas that need attention. In the process of extracting the differential value, using the dense residual connection method for each layer can make the magnified features more complete, so the differential value obtained is more accurate. Next, the joint loss function is introduced to fuse high-frequency information and low-frequency information, which improves the visual effect of the reconstructed image to a certain extent. The experimental results on Set5, Set14, BSD100, and Urban datasets show that our proposed DVDR-SRGAN model is improved in terms of PSNR, SSIM, and LPIPS compared with the Bicubic, SRGAN, ESRGAN, Beby-GAN, and SPSR models.

A novel lightweight multi-dimension feature fusion network for single-image super-resolution reconstruction

A novel lightweight multi-dimension feature fusion network for single-image super-resolution reconstruction

Image Super-Resolution Reconstruction Based on Residual Convolution and Double Attention Mechanism

Currently, single-image super-resolution reconstruction based on deep learning has achieved good results. To address the problems that most networks will have long training time, weak image learning ability and not fully utilizing the high frequency information of images, an image super-resolution reconstruction method based on residual convolution and double attention mechanism is proposed. The model performs deep feature extraction of the image by cascading deep convolutional networks, introduces local residual blocks to solve the model degradation brought by too many network parameters, and embeds the dual attention mechanism module in the residual blocks for adaptive calibration to adjust the feature map weights of each channel and the spatial correlation between features, so as to obtain deep texture detail information and reconstruct the feature image by sub-pixel convolutional layers to up sampling to reconstruct the high-resolution image. In the test sets of Set5 and Set14, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used as evaluation indexes, while comparing SRCNN, FSRCNN and VDSR methods all reconstructed images with better results. The experimental results show that the method can effectively improve the utilization of high-frequency feature information and can increase the reconstruction capability of the images to a certain extent.

Single Image Super-Resolution Reconstruction with Preservation of Structure and Texture Details

In recent years, deep-learning-based single image super-resolution reconstruction has achieved good performance. However, most existing methods pursue a high peak signal-to-noise ratio (PSNR), while ignoring the quality of the structure and texture details, resulting in unsatisfactory performance of the reconstruction results in terms of human subjective perception. To solve this issue, this paper proposes a structure- and texture-preserving image super-resolution reconstruction method. Specifically, two different network branches are used to extract features for image structure and texture details. A dual-coordinate direction perception attention (DCDPA) mechanism is designed to highlight structure and texture features. The attention mechanism fully considers the complementarity and directionality of multi-scale image features and effectively avoids information loss and possible distortion of image structure and texture details during image reconstruction. Additionally, a cross-fusion mechanism is designed to comprehensively utilize structure and texture information for super-resolution image reconstruction, which effectively integrates the structure and texture details extracted by the two branch networks. Extensive experiments verify the effectiveness of the proposed method and its superiority over existing methods.

Single-image super-resolution reconstruction for continuous-wave terahertz imaging systems

Single-image super-resolution reconstruction for continuous-wave terahertz imaging systems

Single image super‐resolution based on sparse representation using edge‐preserving regularization and a low‐rank constraint

Sparse representation-based non-local self-similarity approaches have demonstrated promising performance in single image super-resolution reconstruction. This type of method, however, cannot always effectively preserve the key details of an image, resulting in edge artifacts and local structure blurring. To better preserve the image edge information, in this paper, a sparse representation super-resolution method based on non-local self-similarity is proposed. First, we impose slide window gradient domain guided filtering on both the low-resolution input image and the degraded restored image. Then, we utilize their difference as an edge-preserving regularization term and incorporate this regularization term into the non-local self-similarity-based sparse representation model to build a sparse coding model, which can enhance the restored high-resolution image patches’ detail information. Finally, the iterative threshold algorithm is used to calculate the sparse representation coefficients so that a high-resolution image can be estimated. Furthermore, to explore the potential structures of the subspaces spanned by similar patches, we enforce a low-rank matrix recovery technique on the generated super-resolution image, which can further refine the reconstruction quality. Experimental results prove that the new approach preserves the critical edge structures while suppressing noise and exceeding some popular methods both quantitatively and qualitatively.

Lightweight Multi-Attention Fusion Network for Image Super-Resolution

Single image super-resolution reconstruction (SISR) is one of the important techniques in computer vision and image processing. Most of the existing SISR methods adopt equal processing for different spatial domains and channel domains, resulting in a large amount of computational resources wasted on unimportant features. In order to address these problems, a novel lightweight multi-attention fusion network (LMAFN) is proposed, in which the multiple attention fusion block allocates computational resources more efficiently by capturing the weight information implied by the channel domain and the spatial domain separately, thus effectively reducing the number of parameters. The synthetic channel attention block in the multiple attention fusion block makes full use of inter-channel correlation by introducing both global standard deviation pooling and maximum pooling. Global features are fused through residual linking to alleviate the problem of high frequency information loss. Experimental results on several benchmark datasets show that the proposed method effectively reduces the number of parameters and computational effort without excessive loss of reconstruction performance, and achieves better performance than the compared models.

Super-Resolution Reconstruction of Single Image Combining Bionic Eagle-Eye and Multi-scale

Super-Resolution Reconstruction of Single Image Combining Bionic Eagle-Eye and Multi-scale

Image Super-Resolution Reconstruction Method for Lung Cancer CT-Scanned Images Based on Neural Network.

The super-resolution (SR) reconstruction of a single image is an important image synthesis task especially for medical applications. This paper is studying the application of image segmentation for lung cancer images. This research work is utilizing the power of deep learning for resolution reconstruction for lung cancer-based images. At present, the neural networks utilized for image segmentation and classification are suffering from the loss of information where information passes through one layer to another deep layer. The commonly used loss functions include content-based reconstruction loss and generative confrontation network. The sparse coding single-image super-resolution reconstruction algorithm can easily lead to the phenomenon of incorrect geometric structure in the reconstructed image. In order to solve the problem of excessive smoothness and blurring of the reconstructed image edges caused by the introduction of this self-similarity constraint, a two-layer reconstruction framework based on a smooth layer and a texture layer is proposed for a medical application of lung cancer. This method uses a global nonzero gradient number constrained reconstruction model to reconstruct the smooth layer. The proposed sparse coding method is used to reconstruct high-resolution texture images. Finally, a global and local optimization models are used to further improve the quality of the reconstructed image. An adaptive multiscale remote sensing image super-division reconstruction network is designed. The selective core network and adaptive gating unit are integrated to extract and fuse features to obtain a preliminary reconstruction. Through the proposed dual-drive module, the feature prior drive loss and task drive loss are transmitted to the super-resolution network. The proposed work not only improves the subjective visual effect but the robustness has also been enhanced with more accurate construction of edges. The statistical evaluators are used to test the viability of the proposed scheme.

SUPER RESOLUTION FOR SINGLE SATELLITE IMAGE USING A GENERATIVE ADVERSARIAL NETWORK

Abstract. Inspired by the immense success of deep neural network in image processing and object recognition, learning-based image super resolution (SR) methods have been highly valued and have become the mainstream direction of super resolution research. Base on the recent proposed state-of-art convolution neural network (CNN) super-resolution methods, this paper proposed a generative adversarial network for single satellite image Super Resolution reconstruction. It built on a trained deep residual network to generate preliminary SR images, combined with a discriminative network learns to differentiate preliminary SR images and High resolution samples. The experiments results show that our method can use existing model parameters to refine SR image performance.

Multimodal super-resolution reconstruction of infrared and visible images via deep learning

In this paper, we propose a deep-learning-based infrared-visible images fusion method based on encoder-decoder architecture. The image fusion task is reformulated as a problem of maintaining the structure and intensity ratio of the infrared-visible image. The corresponding loss function is designed to expand the weight difference between the thermal target and the background. In addition, a single image super-resolution reconstruction based on a regression network is introduced to address the issue that traditional network mapping functions are not suitable for natural scenes. The forward generation and reverse regression models are considered to reduce the irrelevant function mapping space and approach the ideal scene data through double mapping constraints. Compared with other state-of-the-art approaches, our experimental results achieve superior performance in terms of both visual effects and objective assessments. In addition, it can stably provide high-resolution reconstruction results consistent with human visual observation while bridging the resolution gap between the infrared-visible images.

Deep learning-based single image super-resolution for low-field MR brain images

Low-field MRI scanners are significantly less expensive than their high-field counterparts, which gives them the potential to make MRI technology more accessible all around the world. In general, images acquired using low-field MRI scanners tend to be of a relatively low resolution, as signal-to-noise ratios are lower. The aim of this work is to improve the resolution of these images. To this end, we present a deep learning-based approach to transform low-resolution low-field MR images into high-resolution ones. A convolutional neural network was trained to carry out single image super-resolution reconstruction using pairs of noisy low-resolution images and their noise-free high-resolution counterparts, which were obtained from the publicly available NYU fastMRI database. This network was subsequently applied to noisy images acquired using a low-field MRI scanner. The trained convolutional network yielded sharp super-resolution images in which most of the high-frequency components were recovered. In conclusion, we showed that a deep learning-based approach has great potential when it comes to increasing the resolution of low-field MR images.

Dual U-Net residual networks for cardiac magnetic resonance images super-resolution

Background and objectiveHeart disease is a vital disease that has threatened human health, and is the number one killer of human life. Moreover, with the added influence of recent health factors, its incidence rate keeps showing an upward trend. Today, cardiac magnetic resonance (CMR) imaging can provide a full range of structural and functional information for the heart, and has become an important tool for the diagnosis and treatment of heart disease. Therefore, improving the image resolution of CMR has an important medical value for the diagnosis and condition assessment of heart disease. At present, most single-image super-resolution (SISR) reconstruction methods have some serious problems, such as insufficient feature information mining, difficulty to determine the dependence of each channel of feature map, and reconstruction error when reconstructing high-resolution image. MethodsTo solve these problems, we have proposed and implemented a dual U-Net residual network (DURN) for super-resolution of CMR images. Specifically, we first propose a U-Net residual network (URN) model, which is divided into the up-branch and the down-branch. The up-branch is composed of residual blocks and up-blocks to extract and upsample deep features; the down-branch is composed of residual blocks and down-blocks to extract and downsample deep features. Based on the URN model, we employ this a dual U-Net residual network (DURN) model, which combines the extracted deep features of the same position between the first URN and the second URN through residual connection. It can make full use of the features extracted by the first URN to extract deeper features of low-resolution images. ResultsWhen the scale factors are 2, 3, and 4, our DURN can obtain 37.86 dB, 33.96 dB, and 31.65 dB on the Set5 dataset, which shows (i) a maximum improvement of 4.17 dB, 3.55 dB, and 3.22dB over the Bicubic algorithm, and (ii) a minimum improvement of 0.34 dB, 0.14 dB, and 0.11 dB over the LapSRN algorithm. ConclusionComprehensive experimental study results on benchmark datasets demonstrate that our proposed DURN can not only achieve better performance for peak signal to noise ratio (PSNR) and structural similarity index (SSIM) values than other state-of-the-art SR image algorithms, but also reconstruct clearer super-resolution CMR images which have richer details, edges, and texture.

Dual Learning-Based Graph Neural Network for Remote Sensing Image Super-Resolution

High-resolution (HR) remote sensing imagery plays a critical role in remote sensing image interpretation, and single image super-resolution (SISR) reconstruction technology is becoming increasingly valuable and significant. The state-of-the-art deep-learning-based SISR methods have demonstrated remarkable advantages, while reconstructing complex texture details still remains a big challenge. Besides, as a typical ill-posed inverse problem, how to determine the optimal solution is another important topic. To address these problems, in this work, a dual learning-based graph neural network (DLGNN) is proposed, in which the GNN is utilized to consider the self-similarity patches in remote sensing imagery by aggregating cross-scale neighboring feature patches, and dual learning strategy is adopted to refine the reconstruction results by constraining the mapping process in terms of the loss function, transferring the typical ill-posed problem to a well-posed one. Abundant experiments on 3K VEHICLE_SR datasets and Massachusetts Roads demonstrate the validity and outstanding performance for remote sensing image super-resolution tasks compared with other state-of-the-art super-resolution construction methods. Code is available at https://github.com/CUG-RS/DLGNN.

Single image super-resolution reconstruction of remotely sensed images using MRF-2D phase congruency model

Recently, the deep learning (DL)-based solution for single image super-resolution reconstruction has been extensively used. Though these methods have shown exceptional results, the computational requirement is very high and requires high-end graphic processors. Motivated to establish an efficient method without such requirement, we propose a modified Markov random field (MRF) model and two-dimensional (2D) phase congruency-based single-image super-resolution reconstruction method for remote sensing images. The 2D phase congruency-based feature extraction method is used to compute features such as edge and texture maps to achieve higher accuracy in finding similar example patches in feature space. To address an essential aspect of successful texture reconstruction, we have incorporated a texture prior for the computation of joint probability in our work. Image Euclidean distance (Ieuc) is integrated to achieve higher accuracy in finding the similarity between image patches in feature space and modeling compatibility functions. The experimental results demonstrate that the results of the proposed method are at par with the DL-based methods and outperform other state-of-the-art methods in perceptual quality as well as peak signal-to-noise ratio and structural similarity index parameters. Moreover, it shows significant improvement in the texture regions while reconstructing sharper edges for ×4 upscaling.