LR Image Research Articles

Enhanced collapsible linear blocks for arbitrary sized image super-resolution

AbstractImage up-scaling and super-resolution (SR) techniques have been a hot research topic for many years due to its large impact in the field of medical imaging, surveillance etc. Especially single image super-resolution (SISR) become very popular because of the fast development of deep convolution neural network (DCNN) and the low requirement on the input. They are achieving outstanding performance. However, there are still problems in the state-of-the-art works, especially from two perspectives: 1. failed at exploiting the hierarchical characteristics from the input, resulting in loss of information and artifacts in the final high resolution (HR) image; 2. failed to handle arbitrary-sized images; the existing research works are focused on fixed size input images. To address these challenges, this paper proposed a residual dense network (RDN) and multi-scale sub-pixel convolution network (MSSPCN) which are integrated into a Collapsible Linear Block Super Efficient Super-Resolution (SESR) network. The RDNs aims to tackle the first challenge, carrying the hierarchical features from end-to-end. An adaptive cropping strategy (ACS) technique is introduced before feature extraction targeting at the image size challenge. The novelty of this work is extracting the hierarchical features and integrating RDNs with MSSPCNs. The proposed network can upscale any arbitrary-sized image (1080p) to ×2 (4K) and ×4 (8K). To secure ground truth for evaluation, this paper follows the opposite flow, generating the input LR images by down-sampling the given HR images (ground truth). To evaluate the performance, the proposed algorithm is compared with eight state-of-the-art algorithms, both quantitatively and qualitatively. The results are verified on six benchmark datasets. The extensive experiments justify that the proposed architecture performs better than other methods and upscales the images satisfactorily.

Text-Enhanced Scene Image Super-Resolution via Stroke Mask and Orthogonal Attention

Low-resolution text images are very commonplace in real life and their information is hard to be extracted by using existing text recognition methods only. Although this problem can be solved by introducing super-resolution (SR) techniques, most existing SR methods fail to process stroke regions and background regions of input text images distinctively. In this paper, we propose a text-specific super-resolution network named Text Enhanced Attention Network (TEAN) to solve this problem. First of all, we compensate for disadvantages of traditional thresholding mask operation proposed in Text Super-Resolution Network (TSRN) by utilizing deep-learning based semantic segmentation method to get correct masks as prior semantic information and propose a Text-Segmented-Contextual-Attention (TSCA) branch on the basis of them. Besides, we design an Orthogonal Contextual Attention Module (OCAM) working with TSCA to implicitly enhance stroke regions of LR images. Secondly, to effectively fuse shallow features and deep features of SR model, we propose a convolutional structure named Weight Balanced Fusion Module (WBFM) to improve traditional feature fusion methods of SR network. Finally, extensive experiments on TextZoom dataset demonstrate that the proposed network can improve the recognition accuracy of text images on existing text recognition models. Using TEAN to process low-resolution text images improves the recognition accuracy by 25.4% on CRNN, by 17.4% on ASTER, by 17.3% on MORAN, by 20.7% on NRTR, by 17.3% on SAR and by 15.9% on MASTER compared with directly recognizing them, which attains competitive performances against state-of-the-art methods. Furthermore, cross-dataset experiments on IC15_2077 demonstrate that TEAN is helpful for scene text recognition task, especially for low-resolution images even with the cross-domain issue.

PSRGAN: Perception-Design-Oriented Image Super Resolution Generative Adversarial Network

Among recent state-of-the-art realistic image super-resolution (SR) intelligent algorithms, generative adversarial networks (GANs) have achieved impressive visual performance. However, there has been the problem of unsatisfactory perception of super-scored pictures with unpleasant artifacts. To address this issue and further improve visual quality, we proposed a perception-design-oriented PSRGAN with double perception turbos for real-world SR. The first-perception turbo in the generator network has a three-level perception structure with different convolution kernel sizes, which can extract multi-scale features from four 14 size sub-images sliced by original LR image. The slice operation expands adversarial samples to four and could alleviate artifacts during GAN training. The extracted features will be eventually concatenated in later 3 × 2 upsampling processes through pixel shuffle to restore SR image with diversified delicate textures. The second-perception turbo in discriminators has cascaded perception turbo blocks (PTBs), which could further perceive multi-scale features at various spatial relationships and promote the generator to restore subtle textures driven by GAN. Compared with recent SR methods (BSRGAN, real-ESRGAN, PDM_SR, SwinIR, LDL, etc.), we conducted an extensive test with a ×4 upscaling factor on various datasets (OST300, 2020track1, RealSR-Canon, RealSR-Nikon, etc.). We conducted a series of experiments that show that our proposed PSRGAN based on generative adversarial networks outperforms current state-of-the-art intelligent algorithms on several evaluation metrics, including NIQE, NRQM and PI. In terms of visualization, PSRGAN generates finer and more natural textures while suppressing unpleasant artifacts and achieves significant improvements in perceptual quality.

Super-Resolution Reconstruction Based on Adaptive Weight Adjustment

In image super-resolution, the existing convolution neural network methods increase the number of network layers and filters to achieve better performance, and seldom consider the influence of different branches in feature extraction on the reconstruction effect, which leads to the problems of blurred details and unclear visual perception. Therefore, we propose an adaptive weight adjustment super-resolution (AWSR) reconstruction model in this paper. The model includes Shallow Feature Extraction (SFE), Information Extraction Enhancement Block (IDEB) and Reconstruction Block (RB). IDEB composed of Adaptive Weight Blocks (AWB) and Channel Linking Layers (CLL) learns a deeper mapping relationship between LR image and HR image by adaptively adjusting the proportions of different branches. It not only saves computational cost, but also improves the expression ability of the model. Meanwhile, the performance of the model is further improved by dimension change in the up-sample block. Especially, the image edge and texture reconstruction effects are obviously improved. Compared with SRNHARB algorithm proposed in 2021, the PSNR values are increased by 0.23[Formula: see text]dB, 0.19[Formula: see text]dB and 0.02[Formula: see text]dB at [Formula: see text] on the Set5 dataset. Moreover, the proposed model has a strong generalization ability, and the reconstructed SR images can achieve satisfactory results.

Unpaired image super-resolution using a lightweight invertible neural network

Unpaired image super-resolution (SR) has recently attracted considerable attention in the unsupervised SR community. In contrast to supervised SR, existing unpaired SR methods inevitably resort to the generative adversarial network (GAN) to explore data distribution on the given HR and unpaired LR dataset. Nevertheless, predominant strategies often strive for sophisticated network structures or training pipelines, making them intractable to apply in real-world scenarios. In this work, a lightweight invertible neural network (INN) is proposed for unpaired SR to alleviate this limitation. Specifically, we regard image degradation and SR as a pair of mutually-inverse tasks and replace the two generators in one-stage GAN with INN. Due to the information lossless nature of INN, it is impossible to generate noise in vain during image degradation. We thus design a simple noise injection network to induce realistic noise, thereby simulating real LR images. To further maintain the stability and realism of the noise, we propose to extract the noise prior from the real-world LR image. With extracted noise prior as input, our noise injection network can narrow the gap between the generated noise and the real one, thereby encouraging the degraded images to match the real-world LR domain. Extensive experiments demonstrate that our method achieves comparable performance with other SOTA methods in quantitative and qualitative evaluations while enjoying faster speed and much smaller parameters.

Multimodal-Boost: Multimodal Medical Image Super-Resolution Using Multi-Attention Network With Wavelet Transform.

Multimodal medical images are widely used by clinicians and physicians to analyze and retrieve complementary information from high-resolution images in a non-invasive manner. Loss of corresponding image resolution adversely affects the overall performance of medical image interpretation. Deep learning-based single image super resolution (SISR) algorithms have revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. While the WT divides the LR image into multiple frequency bands, the transferred GAN uses multi-attention and upsample blocks to predict high-frequency components. Additionally, we present a learning method for training domain-specific classifiers as perceptual loss functions. Using a combination of multi-attention GAN loss and a perceptual loss function results in an efficient and reliable performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning. Using two medical datasets, we validate our proposed SR network against existing state-of-the-art approaches and achieve promising results in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

TLWSR: Weakly supervised real‐world scene text image super‐resolution using text label

AbstractScene text image super‐resolution (STISR) has recently received considerable attention. Existing STISR methods are applicable to the situation that all the LR‐HR pairs are available. However, in real‐world scenarios, it is difficult and expensive to collect ground‐truth HR labels and align them with LR images, and thus it is essential to find a way to implement weakly supervised learning. We investigate the STISR problem in the situation that only a subset of HR labels is available and design a weak supervision framework using coarse‐grained text labels named TLWSR, which combines incomplete supervision and inexact supervision. Specifically, a lightweight text recognition network and connectionist temporal classification loss are used to guide the super‐resolution of text images during training. Extensive experiments on the benchmark TextZoom demonstrate that TLWSR generates distinguishable text images and exceeds the fully supervised baseline TSRN in boosting text recognition accuracywith only 50% HR labels available. Meanwhile, TLWSR can be applied to different super‐resolution backbones and significantly improves their performance. Furthermore, TLWSR shows good generalization capability to low‐quality images on scene text recognition benchmarks, which verifies the effectiveness of this framework. To the authors' knowledge, this is the first work exploring the problem of STISR in weakly supervised scenarios.

SsFPN: Scale Sequence (S2) Feature-Based Feature Pyramid Network for Object Detection.

Object detection is a fundamental task in computer vision. Over the past several years, convolutional neural network (CNN)-based object detection models have significantly improved detection accuracyin terms of average precision (AP). Furthermore, feature pyramid networks (FPNs) are essential modules for object detection models to consider various object scales. However, the AP for small objects is lower than the AP for medium and large objects. It is difficult to recognize small objects because they do not have sufficient information, and information is lost in deeper CNN layers. This paper proposes a new FPN model named ssFPN (scale sequence (S2) feature-based feature pyramid network) to detect multi-scale objects, especially small objects. We propose a new scale sequence (S2) feature that is extracted by 3D convolution on the level of the FPN. It is defined and extracted from the FPN to strengthen the information on small objects based on scale-space theory. Motivated by this theory, the FPN is regarded as a scale space and extracts a scale sequence (S2) feature by three-dimensional convolution on the level axis of the FPN. The defined feature is basically scale-invariant and is built on a high-resolution pyramid feature map for small objects. Additionally, the deigned S2 feature can be extended to most object detection models based on FPNs. We also designed a feature-level super-resolution approach to show the efficiency of the scale sequence (S2) feature. We verified that the scale sequence (S2) feature could improve the classification accuracy for low-resolution images by training a feature-level super-resolution model. To demonstrate the effect of the scale sequence (S2) feature, experiments on the scale sequence (S2) feature built-in object detection approach including both one-stage and two-stage models were conducted on the MS COCO dataset. For the two-stage object detection models Faster R-CNN and Mask R-CNN with the S2 feature, AP improvements of up to 1.6% and 1.4%, respectively, were achieved. Additionally, the APS of each model was improved by 1.2% and 1.1%, respectively. Furthermore, the one-stage object detection models in the YOLO series were improved. For YOLOv4-P5, YOLOv4-P6, YOLOR-P6, YOLOR-W6, and YOLOR-D6 with the S2 feature, 0.9%, 0.5%, 0.5%, 0.1%, and 0.1% AP improvements were observed. For small object detection, the APS increased by 1.1%, 1.1%, 0.9%, 0.4%, and 0.1%, respectively. Experiments using the feature-level super-resolution approach with the proposed scale sequence (S2) feature were conducted on the CIFAR-100 dataset. By training the feature-level super-resolution model, we verified that ResNet-101 with the S2 feature trained on LR images achieved a 55.2% classification accuracy, which was 1.6% higher than for ResNet-101 trained on HR images.

High-resolution aeromagnetic map through Adapted-SRGAN: A case study in Québec, Canada

Due to their cost-effectiveness, aeromagnetic data have been acquired for decades to guide mineral exploration. Aeromagnetic map enhancement is immensely useful as it allows dykes, faults, and other geological structures to be highlighted more clearly, therefore assisting the geologist with a better understanding of the geological process. Over the last years, technological improvements allowed increasing the sensitivity of airborne magnetic data acquisition systems and the accuracy of navigation instrumentation, which resulted in higher resolution (HR) maps. Such higher resolution implies close lines of flight, which typically implies smaller area coverage. On the other hand, the vintage aeromagnetic surveys have high coverage but a much lower resolution. Geological interpretation for regions where only low-resolution aeromagnetic data (LR) are available remains challenging for geologists. Hence, we adapted and trained a deep neural network to address this problem by learning the statistical relations between collocated LR and HR airborne magnetic data. First, we trained a Super-Resolution (SR) network based on an Adapted-SRGAN (ASRGAN) in the source (training) set for mapping LR images to HR images. Second, given this trained network, we generate HR images in the target (test) set where only LR images are available, with a 4 × times resolution increase. We validated the generalization of our model using aeromagnetic maps from several regions of the Québec province, by computing Peak Signal to Noise Ratio (PSNR) and Structural Similarity index (SSIM) between super-resolved GAN outputs with ground truth HR aeromagnetic images. The performance of the proposed approach is compared to bicubic interpolation and conventional SRGAN, with a slight improvement in terms of the SSIM and PSNR but with about 35 percent reduction in the computational time required to train the network.

Fusing Attention Network Based on Dilated Convolution for Superresolution

Deep neural networks with different filters or multiple branches have achieved good performance for single superresolution (SR) in recent years. However, they ignore the high-frequency components of the multiscale context information of the low-resolution image. To solve this problem, we propose a fusing attention network based on dilated convolution (DFAN) for SR. Specifically, we first propose a dilated convolutional attention module (DCAM), which captures multiscale contextual information from different regions of LR images by locking multiple regions with different sizes of receptive fields. Then, we propose a multifeature attention block (MFAB), further focus on high-frequency components of multiscale contextual information, and extract more high-frequency features. Experimental results demonstrate that the proposed DFAN achieves performance improvements in terms of visual quality evaluation and quantitative evaluation.

Enhancement in Skin Cancer Detection using Image Super Resolution and Convolutional Neural Network

Skin cancer has been one of the major worldwide public health issue with more than 1 million cases every year. Skin cancer is classified into three categories: Basal Cell Carcinoma, Melanoma and Squamous Cell Carcinoma [1]. Melanoma is the most critical category of skin cancer with very thin chances of recovery and survival of the patient. Early diagnosis of skin cancer can drastically improve survival rate to as high as 95 percent. This served as a motivation to contribute to this noble cause using technology based solutions. In the process of diagnosis for the disease, the process is divided into four basic components: image processing including hair removal, noise removal, sharpening, and increasing the resolution of the image given to the skin dimension. Recent developments in identification of skin cancer technology uses machine learning and in-depth based reading segmentation algorithms. Most used algorithms are: InceptionV3, ResNet, VGGNet. This paper suggests an artificial skin cancer screening process using techniques like image processing and machine learning. Image super-resolution (ISR) techniques recreate an image with high resolution or sequence from visual LR images. An approach using deep learning on the Image super resolution was used to boost the accuracy of the convolutional neural network model. This model was developed using the Keras backend and tested the model by modifying the layers of neural network which are used for training. The model is built on publicly sourced dataset from the ISIC data archives.

The Best of Both Worlds: A Framework for Combining Degradation Prediction with High Performance Super-Resolution Networks.

To date, the best-performing blind super-resolution (SR) techniques follow one of two paradigms: (A) train standard SR networks on synthetic low-resolution-high-resolution (LR-HR) pairs or (B) predict the degradations of an LR image and then use these to inform a customised SR network. Despite significant progress, subscribers to the former miss out on useful degradation information and followers of the latter rely on weaker SR networks, which are significantly outperformed by the latest architectural advancements. In this work, we present a framework for combining any blind SR prediction mechanism with any deep SR network. We show that a single lightweight metadata insertion block together with a degradation prediction mechanism can allow non-blind SR architectures to rival or outperform state-of-the-art dedicated blind SR networks. We implement various contrastive and iterative degradation prediction schemes and show they are readily compatible with high-performance SR networks such as RCAN and HAN within our framework. Furthermore, we demonstrate our framework's robustness by successfully performing blind SR on images degraded with blurring, noise and compression. This represents the first explicit combined blind prediction and SR of images degraded with such a complex pipeline, acting as a baseline for further advancements.

Super-Resolution of Remote Sensing Images for ×4 Resolution without Reference Images

Sentinel-2 satellites can provide free optical remote-sensing images with a spatial resolution of up to 10 M, but the spatial details provided are not enough for many applications, so it is worth considering improving the spatial resolution of Sentinel-2 satellites images through super-resolution (SR). Currently, the most effective SR models are mainly based on deep learning, especially the generative adversarial network (GAN). Models based on GAN need to be trained on LR–HR image pairs. In this paper, a two-step super-resolution generative adversarial network (TS-SRGAN) model is proposed. The first step is having the GAN train the degraded models. Without supervised HR images, only the 10 m resolution images provided by Sentinel-2 satellites are used to generate the degraded images, which are in the same domain as the real LR images, and then to construct the near-natural LR–HR image pairs. The second step is to design a super-resolution generative adversarial network with strengthened perceptual features, to enhance the perceptual effects of the generated images. Through experiments, the proposed method obtained an average NIQE as low as 2.54, and outperformed state-of-the-art models according to other two NR-IQA metrics, such as BRISQUE and PIQE. At the same time, the comparison of the intuitive visual effects of the generated images also proved the effectiveness of TS-SRGAN.

Pedestrian detection using a translation-invariant wavelet residual dense super-resolution.

Pedestrian detection is an important research area and technology for car driving, gait recognition, and other applications. Although a lot of pedestrian detection techniques have been introduced, low-resolution imaging devices still exist in real life, so detection in low-resolution images remains a challenging problem. To address this issue, we propose a novel end-to-end Translation-invariant Wavelet Residual Dense Super-Resolution (TiWRD-SR) method to upscale LR images to SR images and then use Yolov4 for detection to address the low detection problem performance on low-resolution images. To make the enlarged SR image not only effectively distinguish the foreground and background of images but also highlight the characteristic structure of pedestrians, we decompose the image into low-frequency and high-frequency parts by stationary wavelet transform (SWT). The high- and low-frequency sub-images are trained through different network structures so that the network can reconstruct the high-frequency image edge information and the low-frequency image structure in a more detailed manner. In addition, a high-to-low branch information transmission (H2LBIT) is proposed to import high-frequency image edge information into the low-frequency network to make the reconstructed low-frequency structure more detailed. In addition, we also propose a novel loss function, which enables the SR network to focus on the reconstruction of image structure in the network by the characteristics of wavelet decomposition, thereby improving its detection performance. The experimental results indicate that the proposed TiWRD-SR can effectively improve detection performance.

Dynamic Depth-Aware Network for Endoscopy Super-Resolution

Endoscopy super-resolution (SR) plays an important role in improving diagnostic results and reducing the misdiagnosis rate. Even though recent studies have investigated the SR for endoscopy, these methods apply equal importance to the whole image and do not consider the relationship among pixels, especially the depth information, which can provide diagnosis-related information for clinicians. To address this problem, we propose a dynamic depth-aware network for endoscopy super-resolution, which represents the first effort to comprehensively integrate the depth information to the SR task for endoscopic images. It includes a depth-wise feature extracting branch (DW-B) and a depth-guided SR branch (DGSR-B). The DW-B aims to extract the representative feature for each depth level (i.e. depth matrix) further to provide auxiliary information and guide the super-resolution of texture under different depth levels. In DGSR-B, a depth-guided block (DGB) consisting of depth-focus normalization (DFN) is introduced to inject both the depth matrix and depth map into the LR image feature, so as to guide the image generation for each depth region. To adaptively super-resolve the regions under different depth levels, we devise a dynamic depth-aware loss to assign different trainable weights to each region for SR optimization. Extensive experiments have been conducted on two main publicly available datasets, i.e., the Kvasir dataset and the EndoScene dataset, and the superior performance verifies the effectiveness of our method for SR task and polyp segmentation. Source code is to be released.

Optimized complex object classification model: reconstructing the ISAR image of a hypersonic vehicle covered with a plasma sheath using a U-WGAN-GP framework

ABSTRACT This study aims to apply generative adversarial networks (GANs) to the effective classification of high/low resolution (HR/LR) image pairs obtained via inverse synthetic aperture radar (ISAR) for hypersonic objects covered with a plasma sheath. We propose a classification training model based on a Wasserstein GAN with a gradient penalty (U-WGAN-GP) framework, wherein a U-Net with an excellent jump connection structure is used as a generator, and a VGG-Net with high robustness is used as a discriminator, to support the reliable classification of HR/LR ISAR image pairs for the enhancement of ISAR image resolution. The WGAN-GP provides a shortcut for the gradient propagation of network parameters during the training stage through the stable encoder and decoder structure of U-Net. It inherits the echo intensity variation and position distribution characteristics of the scattering points between hypersonic objects in LR images and effectively transplants them into the generated super-resolution ISAR images, and it establishes end-to-end mapping between the HR/LR ISAR images. Moreover, the VGG-Net ensures that the generated super-resolution images are stable, controllable, and undistorted. In addition, the WGAN-GP structure combines the content and adversarial losses, optimizes the generator loss function, and uses the GP method to provide a stable and continuous training process. Finally, a traditional VGG-16 network classifier is added at the end of the training model. The proposed model is applied to an HR/LR image pair dataset of hypersonic objects. Compared with existing methods, the proposed method improved the classification and recognition accuracy from 54.4% to 81.8%.

APPLICATION OF PRE-TRAINED REAL-WORLD SUPER RESOLUTION MODELS TO OPTICAL SATELLITE IMAGE

Abstract. The single image super-resolution (SISR) technique refers to improving the resolution over the original image. In recent years, we use deep learning-based convolutional neural networks to improve the spatial resolution of images more reasonably. To train such deep learning models, we use training samples consisting of the original HR images and LR images obtained by bicubic downsampling. However, this method of training data using downsampling has a negative impact when applying the trained model to real images. That is because the downsampling function that occurs in the real image is unknown, and the hypothetically created LR image does not represent the resolution degradation that can realistically occur. Therefore, SISR methods that use realistic degradation called real-world super-resolution (RWSR) have been proposed. In this paper, we investigate how such RWSR methods using realistic degradation affect the SISR performance of satellite images. The results of applying the trained model to optical satellite images show that the RWSR method is not the most effective way to handle optical satellite images when compared to the deep learning method without modeling the degradation. In particular, we showed that the effect of RWSR with not only upsampling but also noise and blur removal is significant in the visibility of optical satellite images. Moreover, pre-trained RWSR models can be an aid in visually deciphering ground objects.

Surgical biomicroscopy-guided intra-operative optical coherence tomography (iOCT) image super-resolution

PurposeIntra-retinal delivery of novel sight-restoring therapies will require the precision of robotic systems accompanied by excellent visualisation of retinal layers. Intra-operative Optical Coherence Tomography (iOCT) provides cross-sectional retinal images in real time but at the cost of image quality that is insufficient for intra-retinal therapy delivery.This paper proposes a super-resolution methodology that improves iOCT image quality leveraging spatiotemporal consistency of incoming iOCT video streams.MethodsTo overcome the absence of ground truth high-resolution (HR) images, we first generate HR iOCT images by fusing spatially aligned iOCT video frames. Then, we automatically assess the quality of the HR images on key retinal layers using a deep semantic segmentation model. Finally, we use image-to-image translation models (Pix2Pix and CycleGAN) to enhance the quality of LR images via quality transfer from the estimated HR domain.ResultsOur proposed methodology generates iOCT images of improved quality according to both full-reference and no-reference metrics. A qualitative study with expert clinicians also confirms the improvement in the delineation of pertinent layers and in the reduction of artefacts. Furthermore, our approach outperforms conventional denoising filters and the learning-based state-of-the-art.ConclusionsThe results indicate that the learning-based methods using the estimated, through our pipeline, HR domain can be used to enhance the iOCT image quality. Therefore, the proposed method can computationally augment the capabilities of iOCT imaging helping this modality support the vitreoretinal surgical interventions of the future.

Closed-Loop Residual Attention Network for Single Image Super-Resolution

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

Multi-level U-net network for image super-resolution reconstruction

Deep neural networks have shown better effects for super-resolution. However, it is difficult to extract multi-level features of LR images to reconstruct more clear images. To solve this problem, we present a multi-level U-Net network (MUN) for image super-resolution reconstruction. Specifically, we present a multi-level U-Net residual structure, which is composed of two different levels U-Net structures, to extract multi-level features of LR images. Meanwhile, we present a multi-scale residual block, which extracts multi-level features by dual-branch convolutional layers with different kernels and uses long and short skip connections to bypass a large amount of low-frequency information. Extensive experimental results demonstrate that our presented MUN outperforms other state-of-the-art super-resolution methods.