Super-resolution Network Research Articles

A Two-Stage Three-Dimensional Attention Network for Lightweight Image Super-Resolution

In recent years, single image super-resolution (SISR) methods using convolutional neural networks (CNN) have achieved satisfactory performance. Nevertheless, the large model scale and the slow inference speed of these methods greatly limit the application scenarios. In this paper, we propose a two-stage three-dimensional attention network (ATTNet) for lightweight image super-resolution. First, we put forward the spatial feature encoder–decoder (SFE-D) with a spatial attention mechanism. Next, the channel transposed attention module (CTAM) with a channel self-attention mechanism is designed. Both the modules are used for fine feature extraction in the low-resolution stage. Finally, the content-based pixel recombination module (CPRM) is proposed to reconstruct the detailed content with a joint attention mechanism in the high-resolution stage. According to our experimental results, significant performance in terms of the quantitative metrics and the subjective visual quality can be achieved on average compared with the state-of-the-art lightweight SISR algorithms.

Super-Resolving Face Image by Facial Parsing Information

Face super-resolution is a technology that transforms a low-resolution face image into the corresponding high-resolution one. In this paper, we build a novel parsing map guided face super-resolution network which extracts the face prior (i.e., parsing map) directly from low-resolution face image for the following utilization. To exploit the extracted prior fully, a parsing map attention fusion block is carefully designed, which can not only effectively explore the information of parsing map, but also combines powerful attention mechanism. Moreover, in light of that high-resolution features contain more precise spatial information while low-resolution features provide strong contextual information, we hope to maintain and utilize these complementary information. To achieve this goal, we develop a multi-scale refine block to maintain spatial and contextual information and take advantage of multi-scale features to refine the feature representations. Experimental results demonstrate that our method outperforms the state-of-the-arts in terms of quantitative metrics and visual quality. The source codes will be available at.

A novel person re-identification network to address low-resolution problem in smart city context

We argue that accurate person re-identification is a vital problem for urban public monitoring systems in the smart city context. Since images captured from different cameras have arbitrary resolutions resulting in resolution mismatch, this work proposes a model that takes arbitrary images and converts them to a pre-defined fixed resolution. The model then passes the images to a super-resolution network, producing high-resolution images. We employ a feedback network to generate more realistic super-resolution images, which are fed to the re-identification network to acquire a unique descriptor to disclose the person’s identity. We outperformed in all measures against other state-of-the-art methods.

A Lightweight Low-dose PET Image Super-resolution Reconstruction Method based on Convolutional Neural Network.

PET imaging is one of the most widely used neurological disease screening and diagnosis techniques. Since PET involves the radiation and tolerance of different people, the improvement that has always been focused on is to cut down radiation, in the meantime, ensuring that the generated images with low-dose tracer and generated images with standard-dose tracer have the same details of images. We propose a lightweight low-dose PET super-resolution network (SRPET-Net) based on a convolutional neural network. In this research, We propose a method for accurately recovering highresolution (HR) PET images from low-resolution (LR) PET images. The network learns the details and structure of the image between low-dose PET images and standard-dose PET images and, afterward, reconstructs the PET image by the trained network model. The experiments indicate that the SRPET-Net can achieve a superior peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) values. Moreover, our method has less memory consumption and lower computational cost. In our follow-up work, the technology can be applied to medical imaging in many different directions.

Adaptive Filter in Single Image SRGAN

Abstract: SRGAN is a generative adversarial network for super-resolution of a single picture. In this paper an adaptive filter in single image SRGAN which is a Super-Resolution Generative Adversarial Network (SRGAN), is a deep learning-based approach that is used to generate high-resolution images from low-resolution ones. The SRGAN model is based on the Generative Adversarial Network (GAN) architecture, which consists of two deep neural networks: a generator network and a discriminator network. The generator network takes a low-resolution image as input and tries to generate a high-resolution image that is similar to the original high-resolution image. The discriminator network, on the other hand, tries to distinguish between the high-resolution images generated by the generator network and the original high-resolution images. The SRGAN model has been shown to be very effective in generating high-quality, realistic images from low-resolution inputs, and it has been applied in various applications, including image and video upscaling, medical imaging, and satellite imaging

HASR-TAI: Hybrid model-based interpretable network and super-resolution network for thermoacoustic imaging

Microwave induced thermoacoustic tomography has shown promise for noninvasive and non-ionizing early tumor detection. Nowadays, thermoacoustic reconstruction methods based on deep learning have achieved good and time-efficient results. However, both deep learning methods based on the initial thermoacoustic image and end-to-end methods lack interpretability due to the black-box property of neural networks. In this Letter, we propose an interpretable end-to-end network structure comprising an unrolling part and a super-resolution part. In the unrolling part, a deep unfolding network interprets the iterations of the model-based algorithm based on compressed sensing as layers of the network. Subsequently, a fast and efficient super-resolution neural network maps the low-resolution image into the super-resolution space. Two breast models with different sizes of tumor targets are used for validation. By comparing with the traditional method and the deep learning method, the proposed method demonstrates superior performance in image quality and imaging time. Moreover, the parameters in the network hold physical significance, offering the potential for the interpretable end-to-end network in thermoacoustic imaging.

Depth super-resolution from explicit and implicit high-frequency features

Guided depth super-resolution aims at using a low-resolution depth map and an associated high-resolution RGB image to recover a high-resolution depth map. However, restoring precise and sharp edges near depth discontinuities and fine structures is still challenging for state-of-the-art methods. To alleviate this issue, we propose a novel multi-stage depth super-resolution network, which progressively reconstructs HR depth maps from explicit and implicit high-frequency information. We introduce an efficient transformer to obtain explicit high-frequency information. The shape bias and global context of the transformer allow our model to focus on high-frequency details between objects, i.e., depth discontinuities, rather than texture within objects. Furthermore, we project the input color images into the frequency domain for additional implicit high-frequency cues extraction. Finally, to incorporate the structural details, we develop a fusion strategy that combines depth features and high-frequency information in the multi-stage-scale framework. Exhaustive experiments on the main benchmarks show that our approach establishes a new state-of-the-art. Code will be publicly available at https://github.com/wudiqx106/DSR-EI.

Low-resolution object detection via a lightweight super-resolution network

Low-resolution object detection via a lightweight super-resolution network

Global attention guided multi-scale network for face image super-resolution

Global attention guided multi-scale network for face image super-resolution

MR image reconstruction using iterative up and downsampling network

Magnetic resonance imaging (MRI) is one of the most significant modalities in medical imaging. It suffers from rather lengthy acquisition times, which lead to prolonged patient restriction, patient discomfort, and imaging artifacts. Hence, satisfactory MRI reconstruction from undersampled data sequences constitutes an important research problem. With the advances in deep learning (DL), a plethora of new models have been proposed to solve the MRI reconstruction problem using deep networks. On the other hand, single image super-resolution (SR) is another well-studied field that benefited from the success of DL, and it has applications in various imaging modalities. SR is the process of recovering a high-resolution image from a low-resolution image. SR models work on low-resolution images to recover missing details. MR image reconstruction on the other hand is a battle to eliminate the aliasing artifacts which originate from data downsampling. The motivation for the proposed work is based on the premise that SR approaches can possibly get adapted to MR image reconstruction. Hence in this study, inspired by the great success of deep SR networks, we customize an architecture introduced in SR setting to MRI reconstruction. This novel approach uses the iterative up and downsampling framework labeled as Iterative Up and Down Network (IUDN) for MRI reconstruction. We design two variants of the proposed network with different number of scale factors. We present extensive simulations for the proposed architecture using multiple k-space undersampling ratios. The simulation results indicate cutting edge MRI reconstruction performance for the proposed models. The networks were trained on fastMRI dataset and tested on both fastMRI and IXI datasets to show the robustness of the method. The proposed models achieved improved results in terms of PSNR and visual quality of the reconstructed image when compared to some recent state-of-the-art solutions for MRI reconstruction.

MFFN: Multi‐path feedback fusion network for lightweight image super resolution

AbstractRecently, convolutional neural network (CNN) has shown great power in single image super resolution (SISR) reconstruction, and achieving significant improvements over traditional methods. Despite the great success of these CNN‐based methods, direct application of these methods to some edge devices is impractical due to the large computational overhead required. To address this problem, a novel, lightweight SISR network focusing on speed and accuracy, called the multi‐path feedback fusion network (MFFN), has been designed in this paper. Specifically, in order to extract features more effectively, the authors propose a novel fusion attention feedback block (FAFB) as the main building block of MFFN. The FAFB consists of a backbone branch and several hierarchical branches. The backbone branch is composed of stacked enhanced pixel attentional blocks (EPAB), which are responsible for incremental deep feature learning on the feature map. And the hierarchical branches are responsible for extracting feature maps with different sizes of receptive fields and fusing these feature maps with the features extracted from the trunk branches to achieve multi‐scale feature learning, which the authors refer to this design as the multi‐scale fusion block (MSFB). Extra enhancement information (EIE) is added to each EPAB input, which enables the backbone branch to learn more effectively. On the other hand, the outputs of the cascade branches are further complemented by an additional feedback fusion enhancement block (FFEB) before being fused with the output of the trunk branches to achieve more comprehensive and accurate feature learning. Numerous experiments have shown that MFFN achieves higher accuracy than other state‐of‐the‐art methods on benchmark test sets.

Real-Time Video Super-Resolution with Spatio-Temporal Modeling and Redundancy-Aware Inference.

Video super-resolution aims to generate high-resolution frames from low-resolution counterparts. It can be regarded as a specialized application of image super-resolution, serving various purposes, such as video display and surveillance. This paper proposes a novel method for real-time video super-resolution. It effectively exploits spatial information by utilizing the capabilities of an image super-resolution model and leverages the temporal information inherent in videos. Specifically, the method incorporates a pre-trained image super-resolution network as its foundational framework, allowing it to leverage existing expertise for super-resolution. A fast temporal information aggregation module is presented to further aggregate temporal cues across frames. By using deformable convolution to align features of neighboring frames, this module takes advantage of inter-frame dependency. In addition, it employs a hierarchical fast spatial offset feature extraction and a channel attention-based temporal fusion. A redundancy-aware inference algorithm is developed to reduce computational redundancy by reusing intermediate features, achieving real-time inferring speed. Extensive experiments on several benchmarks demonstrate that the proposed method can reconstruct satisfactory results with strong quantitative performance and visual qualities. The real-time inferring ability makes it suitable for real-world deployment.

A lightweight super-resolution network with skip-connections.

In some hospitals in remote areas, due to the lack of MRI scanners with high magnetic field intensity, only low-resolution MRI images can be obtained, hindering doctors from making correct diagnoses. In our study, higher-resolution images were obtained through low-resolution MRI images. Moreover, as our algorithm is a lightweight algorithm with a small number of parameters, it can be carried out in remote areas under the condition of the lack of computing resources. Moreover, our algorithm is of great clinical significance in providing references for doctors' diagnoses and treatment in remote areas. We compared different super-resolution algorithms to obtain high-resolution MRI images, including SRGAN, SPSR, and LESRCNN. A global skip connection was applied to the original network of LESRCNN to use global semantic information to get better performance. Experiments reported that our network improved SSMI by 0.8% and also achieved an obvious increase in PSNR, PI, and LPIPS compared to LESRCNN in our dataset. Similar to LESRCNN, our network has a very short running time, a small number of parameters, low time complexity, and low space complexity while ensuring high performance compared to SRGAN and SPSR. Five MRI doctors were invited for a subjective evaluation of our algorithm. All agreed on significant improvements and that our algorithm could be used clinically in remote areas and has great value. The experimental results demonstrated the performance of our algorithm in super-resolution MRI image reconstruction. It allows us to obtain high-resolution images in the absence of high-field intensity MRI scanners, which have great clinical significance. The short running time, a small number of parameters, low time complexity, and low space complexity ensure that our network can be used in grassroots hospitals in remote areas that lack computing resources. We can reconstruct high-resolution MRI images in a short time, thus saving time for patients. Our algorithm can be biased towards practical applications; however, doctors have affirmed the clinical value of our algorithm.

A Super-Resolution Network for High-Resolution Reconstruction of Landslide Main Bodies in Remote Sensing Imagery Using Coordinated Attention Mechanisms and Deep Residual Blocks

The lack of high-resolution training sets for intelligent landslide recognition using high-resolution remote sensing images is a major challenge. To address this issue, this paper proposes a method for reconstructing low-resolution landslide remote sensing images based on a Super-Resolution Generative Adversarial Network (SRGAN) to fully utilize low-resolution images in the process of constructing high-resolution landslide training sets. First, this paper introduces a novel Enhanced Depth Residual Block called EDCA, which delivers stable performance compared to other models while only slightly increasing model parameters. Secondly, it incorporates coordinated attention and redesigns the feature extraction module of the network, thus boosting the learning ability of image features and the expression of high-frequency information. Finally, a residual stacking-based landslide remote sensing image reconstruction strategy was proposed using EDCA residual blocks. This strategy employs residual learning to enhance the reconstruction performance of landslide images and introduces LPIPS for evaluating the test images. The experiment was conducted using landslide data collected by drones in the field. The results show that compared with traditional interpolation algorithms and classic deep learning reconstruction algorithms, this approach performs better in terms of SSIM, PSNR, and LPIPS. Moreover, the network can effectively handle complex features in landslide scenes, which is beneficial for subsequent target recognition and disaster monitoring.

GRAN: ghost residual attention network for single image super resolution

GRAN: ghost residual attention network for single image super resolution

Global Aligned Structured Sparsity Learning for Efficient Image Super-Resolution.

Efficient image super-resolution (SR) has witnessed rapid progress thanks to novel lightweight architectures or model compression techniques (e.g., neural architecture search and knowledge distillation). Nevertheless, these methods consume considerable resources or/and neglect to squeeze out the network redundancy at a more fine-grained convolution filter level. Network pruning is a promising alternative to overcome these shortcomings. However, structured pruning is known to be tricky when applied to SR networks because the extensive residual blocks demand the pruned indices of different layers to be the same. Besides, the principled determination of proper layerwise sparsities remains challenging too. In this paper, we present Global Aligned Structured Sparsity Learning (GASSL) to resolve these problems. GASSL has two major components: Hessian-Aided Regularization (HAIR) and Aligned Structured Sparsity Learning (ASSL). HAIR is a regularization-based sparsity auto-selection algorithm with Hessian considered implicitly. A proven proposition is introduced to justify its design. ASSL is for physically pruning SR networks. Particularly, a new penalty term Sparsity Structure Alignment (SSA) is proposed to align the pruned indices of different layers. With GASSL, we design two new efficient single image SR networks of different architecture genres, pushing the efficiency envelope of SR models one step forward. Extensive results demonstrate the merits of GASSL over other recent counterparts.

Generation of High-Resolution Facial Expression Images Using a Super-Resolution Technique and Self-Supervised Guidance

The recent spread of smartphones and social networking services has increased the means of seeing images of human faces. Particularly, in the face image field, the generation of face images using facial expression transformation has already been realized using deep learning–based approaches. However, in the existing deep learning–based models, only low-resolution images can be generated due to limited computational resources. Consequently, the generated images are blurry or aliasing. To address this problem, we proposed a two-step method to enhance the resolution of the generated facial images by combining a super-resolution network following the generative model, which can be considered a serial model, in our previous work. We further proposed a parallel model that trains a generative adversarial network and a superresolution network through multitask learning. In this paper, we propose a new model that integrates self-supervised guidance encoders into the parallel model to further improve the accuracy of the generated results. Using the peak signalto- noise ratio as an evaluation index, image quality was improved by 0.25 dB for the male test data and 0.28 dB for the female test data compared with our previous multitaskbased parallel model.

Enhanced Compressed Sensing Based Deep Learning Neural Network for Single Image Super Resolution of COVID-19 Using X-Ray Images

Enhanced Compressed Sensing Based Deep Learning Neural Network for Single Image Super Resolution of COVID-19 Using X-Ray Images

SCN: Self-Calibration Network for fast and accurate image super-resolution

SCN: Self-Calibration Network for fast and accurate image super-resolution

Lightweight subpixel sampling network for image super-resolution

Lightweight subpixel sampling network for image super-resolution