Image Super-resolution Algorithm Research Articles

Image super-resolution reconstruction algorithm based on significant network connection-collaborative migration structure

Although deep learning is widely used in the field of image super-resolution reconstruction, the number of network layers and the complexity of deep learning models continue to increase, and the speed at which the difficulty of model training increases has exceeded the speed of network performance improvement. This situation prevents deep learning frameworks from fully utilizing their generalization ability in image super-resolution reconstruction. Moreover, the existing image reconstruction methods have the problem of partial information loss. Therefore, this paper first proposes a structure that simulates the feature extraction function in the visual attention mechanism in the convolutional neural network. We call this a significant network connection. The feature information extracted by the network architecture is more significant, while other information is less significant. This setup has a reduced impact on the resulting image reconstruction effect. Then, a network architecture focusing on collaborative information migration is proposed. This architecture can obtain the implicit domain of the intermediate state of the image domain to be reconstructed, and it can make the two networks use the learned hidden domain during the reconstruction process. The dual networks trained in this way are more symmetrical. This approach can better maintain the common feature information of the reconstructed image and effectively solve the problem of partial feature information loss in the image. The experimental results show that the texture, artificial effects and noise of the reconstructed image obtained by the method proposed in this paper are significantly improved over those of the images produced by other mainstream methods. In addition, the method proposed in this paper exhibits a certain degree of improvement over other deep learning methods in terms of model training speed and feature information retention.

Multi-Scale Factor Image Super-Resolution Algorithm with Information Distillation Network

Deep convolutional neural networks with strong expressive ability have achieved impressive performances in single-image super-resolution algorithms. However, excessive convolutions usually consume high computational cost, which limits the application of super-resolution technology in low computing power devices. Besides, super-resolution of arbitrary scale factor has been ignored for a long time. Most previous researchers have trained a specific network model separately for each factor, and taken the super-resolution of several integer scale factors into consideration. In this paper, we put forward a multi-scale factor network (MFN), which dynamically predicts the weights of the upscale filter by taking the scale factor as input, and generates HR images with corresponding scale factors from the weights. This method is suitable for arbitrary scale factors (integer or non-integer). In addition, we use an information distillation structure to gradually extract multi-scale spatial features. Extensive experiments suggest that the proposed method performs favorably against the state-of-the-art SR algorithms in term of visual quality, PSNR/SSIM evaluation indicators, and model parameters.

A novel image super-resolution algorithm based on multi-scale dense recursive fusion network

With the increasing maturity of convolution neural network (CNN) technology, the image super-resolution reconstruction (SR) method based on CNN is booming and has achieved many remarkable results. Undoubtedly, SR has become the mainstream direction of image reconstruction technology. However, most of the existing SR methods improve the reconstruction performance by increasing the depth of networks, which also increases the number of parameters, number of network computations, and difficulty of training network. To solve the performance complexity dilemma in SR, this paper proposes a network called a multi-scale dense recursive fusion network (MSDRFN). The network is composed of three parts: initial feature extraction module, multi-scale dense fusion group module and recursive reconstruction module. In detail, rough features are first extracted through a shallow feature extraction module, and then are inputted into multi-scale dense fusion blocks (MSDFBs) group. Each MSDFB makes full use of image features in convolution kernels of different sizes to obtain different hierarchical features, and further these output features are inputted into the channel attention mechanism to learn their corresponding weights. All MSDFBs outputs will be restored to high resolution images via the recursive reconstruction module. In addition, the network supplements the information loss with residual learning, which is embodied in one long-jump connection and several short-jump connections. The proposed network is mainly trained in the Pytorch deep learning framework. In comparison experiments on benchmark datasets, the proposed method outperformed the most advanced convolutional methods.

Structure-aware Meta-fusion for Image Super-resolution

There are two main categories of image super-resolution algorithms: distortion oriented and perception oriented. Recent evidence shows that reconstruction accuracy and perceptual quality are typically in disagreement with each other. In this article, we present a new image super-resolution framework that is capable of striking a balance between distortion and perception. The core of our framework is a deep fusion network capable of generating a final high-resolution image by fusing a pair of deterministic and stochastic images using spatially varying weights. To make a single fusion model produce images with varying degrees of stochasticity, we further incorporate meta-learning into our fusion network. Once equipped with the kernel produced by a kernel prediction module, our meta fusion network is able to produce final images at any desired level of stochasticity. Experimental results indicate that our meta fusion network outperforms existing state-of-the-art SISR algorithms on widely used datasets, including PIRM-val, DIV2K-val, Set5, Set14, Urban100, Manga109, and B100. In addition, it is capable of producing high-resolution images that achieve low distortion and high perceptual quality simultaneously.

Algorithms for multi-frame image super-resolution under applicative noise based on deep neural networks

The article describes algorithms for multi-frame image super-resolution, which recover high-resolution images from a sequence of low-resolution images of the same scene under applicative noise. Applicative noise generates local regions of outlying observations in each image and reduces the image resolution. So far, little attention has been paid to this problem. At the same time, the use of deep neural networks is considered to be a promising method of image processing, including multi-frame image super-resolution. The article considers the existing solutions to the problem and suggests a new approach based on using several pre-trained convolutional neural networks and directed acyclic graph neural networks trained by the authors. The developed approach and the algorithms based on this approach involve iterative processing of the input sequence of low-resolution images using different neural networks at different processing stages. The stages include registration of low-resolution images, their segmentation performed in order to determine regions damaged by applicative noise, and transformation performed in order to increase the resolution. The approach combines the strengths of the existing solutions while lacking their drawbacks resulting from the use of approximate mathematical data models required for the synthesis of the image processing algorithms within the statistical theory of solutions. The experimental studies demonstrated that the suggested algorithm is fully functional and allows more accurate recovery of high-resolution images than the existing analogues.

Deep Learning-based Image Super-resolution Algorithms - A Survey

Deep Learning-based Image Super-resolution Algorithms - A Survey

Magnetic Resonance Imaging Features on Deep Learning Algorithm for the Diagnosis of Nasopharyngeal Carcinoma.

The objective of this research was to investigate the application values of magnetic resonance imaging (MRI) features of the deep learning-based image super-resolution reconstruction algorithm optimized convolutional neural network (OPCNN) algorithm in nasopharyngeal carcinoma (NPC) lesion diagnosis. A total of 54 patients with NPC were selected as research objects. Based on the traditional CNN structure, OPCNN was proposed. Besides, MRI processed by the traditional CNN model and the U-net network model was introduced to be analyzed and compared with its algorithm. The used assessment parameters included volume transfer constant (Ktrans), rate constant (Kep), volume fraction (Ve), and apparent diffusion coefficient (ADC). The results showed that the values of Dice coefficient, peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) of the OPCNN algorithm were significantly higher than those of the traditional CNN model and the U-net network model. Meanwhile, the difference was statistically significant (P < 0.05). Ktrans, Kep, and Ve in tumor lesions were significantly higher than those in the healthy side, while the ADC was significantly lower than that in the healthy side (P < 0.05). The sensitivity, specificity, and accuracy of dynamic contrast-enhancement magnetic resonance imaging (DCE-MRI) in the diagnosis of nasopharyngeal carcinoma staging were slightly higher than those in T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI). The diagnostic sensitivity of DCE-MRI was more than 85%, its diagnostic specificity was more than 75%, and its diagnostic accuracy was more than 90%. The AUC area of NPC diagnosed by combination of the three was significantly different from that diagnosed by single T2WI, DWI, and DCE-MRI (P < 0.05). The diagnostic accuracy of MRI based on the OPCNN algorithm for nasopharyngeal carcinoma (93.2%) was significantly higher than that of single MRI (76.4%). In summary, the OPCNN algorithm proposed in this study could improve the quality of MRI images, and the effect was better than the traditional deep learning model, which had the value of clinical promotion. The application value of DCE-MRI in the diagnosis of pathogenic lesions of nasopharyngeal carcinoma was better than conventional MRI. The combined application of T2WI, DWI, and DCE-MRI in the screening of nasopharyngeal carcinoma lesions could greatly improve the diagnostic accuracy of nasopharyngeal carcinoma.

Visible-Assisted Infrared Image Super-Resolution Based on Spatial Attention Residual Network

Infrared images have a wide range of applications in military and civilian fields, including night vision, surveillance, and robotics. However, the most commonly used infrared images are low-resolution (LR), which lack texture details, and existing infrared image super-resolution (SR) algorithms are limited by the lack of spatial information utilization. To solve the above problems, a spatial attention residual network (SAResNet) is proposed. Specifically, the network consists of spatial attention residual block (SARB) with several short skip connections (SSCs). The SARB contains 20 spatial attention blocks (SAB), which adaptively adjusts weights of different spatial regions by considering interdependence between spatial features. Meanwhile, the visible images are considered as complementary sources; thus, a visible-assisted training strategy is designed for the infrared SR process, promoting details preservation. Furthermore, the spatial attention (SA) mechanism is utilized, which focuses more on spatial characteristics of the image and refines the main objects and target boundaries. Experimentally, the proposed method, SAResNet, is compared with existing SR methods, and the effectiveness of the proposed method is demonstrated based on both quantity and quality analyses.

Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and an Objective Metric.

Numerous single image super-resolution (SISR) algorithms have been proposed during the past years to reconstruct a high-resolution (HR) image from its low-resolution (LR) observation. However, how to fairly compare the performance of different SISR algorithms/results remains a challenging problem. So far, the lack of comprehensive human subjective study on large-scale real-world SISR datasets and accurate objective SISR quality assessment metrics makes it unreliable to truly understand the performance of different SISR algorithms. We in this paper make efforts to tackle these two issues. Firstly, we construct a real-world SISR quality dataset (i.e., RealSRQ) and conduct human subjective studies to compare the performance of the representative SISR algorithms. Secondly, we propose a new objective metric, i.e., KLTSRQA, based on the Karhunen-Loéve Transform (KLT) to evaluate the quality of SISR images in a no-reference (NR) manner. Experiments on our constructed RealSRQ and the latest synthetic SISR quality dataset (i.e., QADS) have demonstrated the superiority of our proposed KLTSRQA metric, achieving higher consistency with human subjective scores than relevant existing NR image quality assessment (NR-IQA) metrics. The dataset and the code will be made available at https://github.com/Zhentao-Liu/RealSRQ-KLTSRQA.

Super-Resolution of GF-1 Multispectral Wide Field of View Images via a Very Deep Residual Coordinate Attention Network

GF-1 multispectral wide field of view (WFV) images, with a spatial resolution of 16 m, have been widely used in earth monitoring. However, the spatial details provided by WFV images are not sufficient for many applications. Thus, this letter proposes a novel WFV image super-resolution (SR) algorithm called GFRCAN based on a very deep residual coordinate attention network. To form a very deep network, the residual-in-residual (RIR) structure consisting of several residual groups (RG) with long skip connections is used. Meanwhile, the residual coordinate attention block (RCOAB) and adaptive multi-scale spatial attention module (AMSA) are incorporated to focus on the high-frequency information and multi-scale features adaptive weighted fusion. Besides, the spectral and spatial details of SR images are improved by incorporating peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) into the loss function. Both subjective and objective evaluation results show that the proposed model outperforms the state-of-the-art methods.

Selfie Periocular Verification Using an Efficient Super-Resolution Approach

Selfie-based biometrics has great potential for a wide range of applications since, e.g. periocular verification is contactless and is safe to use in pandemics such as COVID-19, when a major portion of a face is covered by a facial mask. Despite its advantages, selfie-based biometrics presents challenges since there is limited control over data acquisition at different distances. Therefore, Super-Resolution (SR) has to be used to increase the quality of the eye images and to keep or improve the recognition performance. We propose an Efficient Single Image Super-Resolution algorithm, which takes into account a trade-off between the efficiency and the size of its filters. To that end, the method implements a loss function based on the Sharpness metric used to evaluate iris images quality. Our method drastically reduces the number of parameters compared to the state-of-the-art: from 2,170,142 to 28,654. Our best results on remote verification systems with no redimensioning reached an EER of 8.89% for FaceNet, 12.14% for VGGFace, and 12.81% for ArcFace. Then, embedding vectors were extracted from SR images, the FaceNet-based system yielded an EER of 8.92% for a resizing of x2, 8.85% for x3, and 9.32% for x4.

Hybrid Single Image Super-Resolution Algorithm for Medical Images

High-quality medical microscopic images used for diseases detection are expensive and difficult to store. Therefore, low-resolution images are favorable due to their low storage space and ease of sharing, where the images can be enlarged when needed using Super-Resolution (SR) techniques. However, it is important to maintain the shape and size of the medical images while enlarging them. One of the problems facing SR is that the performance of medical image diagnosis is very poor due to the deterioration of the reconstructed image resolution. Consequently, this paper suggests a multi-SR and classification framework based on Generative Adversarial Network (GAN) to generate high-resolution images with higher quality and finer details to reduce blurring. The proposed framework comprises five GAN models: Enhanced SR Generative Adversarial Networks (ESRGAN), Enhanced deep SR GAN (EDSRGAN), Sub-Pixel-GAN, SRGAN, and Efficient Wider Activation-B GAN (WDSR-b-GAN). To train the proposed models, we have employed images from the famous BreakHis dataset and enlarged them by 4× and 16× upscale factors with the ground truth of the size of 256 × 256 × 3. Moreover, several evaluation metrics like Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Structural Similarity Index (SSIM), Multiscale Structural Similarity Index (MS-SSIM), and histogram are applied to make comprehensive and objective comparisons to determine the best methods in terms of efficiency, training time, and storage space. The obtained results reveal the superiority of the proposed models over traditional and benchmark models in terms of color and texture restoration and detection by achieving an accuracy of 99.7433%.

Bayesian Forward-Looking Superresolution Imaging Using Doppler Deconvolution in Expanded Beam Space for High-Speed Platform

Deconvolution technique can be utilized in the forward-looking radar (FLR). However, the forward-looking imaging performance degenerates greatly due to the effect of high-speed movement of the platform. In this article, an efficient Bayesian forward-looking superresolution imaging algorithm based on Doppler deconvolution in expanded beam space is proposed. First, the Doppler phase information caused by the high-speed platform is fully exploited and the Doppler matrix is integrated with the antenna pattern. The Doppler convolution model of the echo signal for forward-looking is derived in this article. Then, the Doppler phase information is adopted to perform the Doppler deconvolution. Moreover, an expanded beam space is constructed to enhance the sparsity of the imaging scene. The complex Gaussian distribution and the Laplace distribution have been used to model the distribution characteristics of noise and targets in the imaging scene, respectively. Finally, based on the Bayesian framework, the forward-looking imaging problem is converted into the convex optimization problem. The performance assessment based on simulated and experimental data, also in comparison to the conventional real beam, truncated singular value decomposition (TSVD), iterative adaptive approach (IAA) methods, has demonstrated the effectiveness of our proposed algorithm under high-speed platform scenarios.

Super-resolution reconstruction of pneumocystis carinii pneumonia images based on generative confrontation network

ObjectivePneumocystis carinii pneumonia, also known as pneumocystis carinii pneumonia (PCP), is an interstitial plasma cell pneumonia caused by pneumocystis spp. It is a conditional lung infectious disease. Because the early and correct diagnosis of PCP has a great influence on the prognosis of patients, the image processing of PCP's high-resolution CT (HRCT) is extremely important. Traditional image super-resolution reconstruction algorithms have difficulties in network training and artifacts in generated images. The super-resolution reconstruction algorithm of generative counter-networks can optimize these two problems well. MethodsIn this paper, the texture enhanced super-resolution generative adversarial network (TESRGAN) is based on a generative confrontation network, which mainly includes a generative network and a discriminant network. In order to improve the quality of image reconstruction, TESRGAN improved the structure of the Super-Resolution Generative Adversarial Network (SRGAN) generation network, removed all BN layers in SRGAN, and replaced the ReLU function with the LeakyReLU function as the nonlinear activation function of the network to avoid the disappearance of the gradient. Experimental resultsThe TESRGAN algorithm in this paper is compared with the image reconstruction results of Bicubic, SRGAN, Enhanced Deep Super-Resolution network (EDSR), and ESRGAN. Compared with algorithms such as SRGAN and EDSR, our algorithm has clearer texture details and more accurate brightness information without extending the running time. Our reconstruction algorithm can improve the accuracy of image low-frequency information. ConclusionThe texture details of the reconstruction result are clearer and the brightness information is more accurate, which is more in line with the requirements of visual sensory evaluation.

Image Compression Algorithm Based On Variational Autoencoder

Variational Autoencoder (VAE), as a kind of deep hidden space generation model, has achieved great success in performance in recent years, especially in image generation. This paper aims to study image compression algorithms based on variational autoencoders. This experiment uses the image quality evaluation measurement model, because the image super-resolution algorithm based on interpolation is the most direct and simple method to change the image resolution. In the experiment, the first step of the whole picture is transformed by the variational autoencoder, and then the actual coding is applied to the complete coefficient. Experimental data shows that after encoding using the improved encoding method of the variational autoencoder, the number of bits required for the encoding symbol stream required for transmission or storage in the traditional encoding method is greatly reduced, and symbol redundancy is effectively avoided. The experimental results show that the image research algorithm using variational autoencoder for image 1, image 2, and image 3 reduces the time by 3332, 2637, and 1470 bit respectively compared with the traditional image research algorithm of self-encoding. In the future, people will introduce deep convolutional neural networks to optimize the generative adversarial network, so that the generative adversarial network can obtain better convergence speed and model stability.

Exploring Super-Resolution for Face Recognition

Biometric recognition is part of many aspects of modern society. With the popularization of smartphones, facial recognition gains space in this environment of biometric technologies. With the diversity of image capture devices, of different brands and qualities, the images will not always be in the ideal standard to be recognized. This article tests and compares different scenarios and situations to assess the results obtained by facial recognition in different environments. For this, the quantity method of data analysis was used. In the first scenario, all images were submitted without changes. In the following, we have the reduction of image resolution, which may or may not be followed by enlargement to the original resolution via bicubic interpolation or through the Image Super-Resolution algorithm, these images can be all, or only that undergo tests. Results indicate that the first scenario obtained the best performance, followed by only the tests images change. The worst performance occurs where the properties of all images are affected. In situations where there is a reduction and enlargement are optional, the enlargement option performs better, so the bicubic enlargement has an advantage over the ISR, the situation in which only the reduction occurs has the worst performance.

RETRACTED: Chen et al. Image Super-Resolution Algorithm Based on Dual-Channel Convolutional Neural Networks. Appl. Sci. 2019, 9, 2316

The published article Image Super-Resolution Algorithm Based on Dual-Channel Convolutional Neural Networks [...]

Single Image Super-Resolution Reconstruction Algorithm with Generative Adversarial Network Based on Multi-Size Convolution and Laplacian Filtering

<p indent="0mm">At present, the image super-resolution reconstruction algorithm using deep learning still faces problems such as insufficient texture perception and insufficient realism of the reconstructed image. In order to improve the quality of the reconstructed image, a single image super-resolution reconstruction algorithm with generative adversarial network based on multi-size convolution and Laplacian filtering is proposed. First, multi-size convolution feature extraction, separable Laplacian filtering and composite residual dense block are used to build a generation network which can extract more comprehensive image information. Secondly, multi-dimensional soft labels are utilized to construct the adversarial network, which can train the generative adversarial network easily and reconstruct image texture richly. Finally, the <italic>L</italic>₁ loss function and the VGG low-level features are taken to obtain the overall features in the pre-training stage and the VGG high-level features, Charbonnier loss, and generative loss are used to make the reconstruction result more meticulously during training. Div2k and Flickr2K are chosen for model training, and Set5 and other data sets are used for testing. The experiment results show that the network size of this algorithm is 40% less than USRNet and other related algorithms, and the perceptual index is 0.76% lower than USRNet on average. The reconstruction result has more details and is more authentic.

URNet: A U-Shaped Residual Network for Lightweight Image Super-Resolution

It is extremely important and necessary for low computing power or portable devices to design more lightweight algorithms for image super-resolution (SR). Recently, most SR methods have achieved outstanding performance by sacrificing computational cost and memory storage, or vice versa. To address this problem, we introduce a lightweight U-shaped residual network (URNet) for fast and accurate image SR. Specifically, we propose a more effective feature distillation pyramid residual group (FDPRG) to extract features from low-resolution images. The FDPRG can effectively reuse the learned features with dense shortcuts and capture multi-scale information with a cascaded feature pyramid block. Based on the U-shaped structure, we utilize a step-by-step fusion strategy to improve the performance of feature fusion of different blocks. This strategy is different from the general SR methods which only use a single Concat operation to fuse the features of all basic blocks. Moreover, a lightweight asymmetric residual non-local block is proposed to model the global context information and further improve the performance of SR. Finally, a high-frequency loss function is designed to alleviate smoothing image details caused by pixel-wise loss. Simultaneously, the proposed modules and high-frequency loss function can be easily plugged into multiple mature architectures to improve the performance of SR. Extensive experiments on multiple natural image datasets and remote sensing image datasets show the URNet achieves a better trade-off between image SR performance and model complexity against other state-of-the-art SR methods.

Research on image super-resolution algorithm based on mixed deep convolutional networks

The existing image processing methods had aimed at the problems of blurred image reconstruction, large noise, and poor visual perception. The improved image super-resolution algorithm based on mixed deep convolutional networks is proposed in the paper. Firstly, the proposed method can shrink the low-resolution image to the specified size in upsampling phase. Secondly, it can extract features from low-resolution images. It sends the extracted initial features into the convolutional coding and decoding structure for image features. Thirdly, the feature extraction and calculation in high-dimensions are performed using dilated convolution in reconstruction layer. The high-resolution image had been reconstructed. The proposed method had been compared with state-of-arts on Set5, Set14, BSD100, and Urban100 datasets. The experimental results can show that the Peak Signal-to-Noise Ratio is increased by some ranges, and the Structural Similarity is increased by some effective percentage points.