Single Image Super-resolution Technique Research Articles

Lightweight Super-Resolution Techniques in Medical Imaging: Bridging Quality and Computational Efficiency.

Medical imaging plays an essential role in modern healthcare, providing non-invasive tools for diagnosing and monitoring various medical conditions. However, the resolution limitations of imaging hardware often result in suboptimal images, which can hinder the precision of clinical decision-making. Single image super-resolution (SISR) techniques offer a solution by reconstructing high-resolution (HR) images from low-resolution (LR) counterparts, enhancing the visual quality of medical images. In this paper, we propose an enhanced Residual Feature Learning Network (RFLN) tailored specifically for medical imaging. Our contributions include replacing the residual local feature blocks with standard residual blocks, increasing the model depth for improved feature extraction, and incorporating enhanced spatial attention (ESA) mechanisms to refine the feature selection. Extensive experiments on medical imaging datasets demonstrate that the proposed model achieves superior performance in terms of both quantitative metrics, such as PSNR and SSIM, and qualitative visual quality compared to existing state-of-the-art models. The enhanced RFLN not only effectively mitigates noise but also preserves critical anatomical details, making it a promising solution for high-precision medical imaging applications.

A novel theoretical analysis on optimal pipeline of multi-frame image super-resolution using sparse coding

Super-resolution is the process of obtaining a high-resolution (HR) image from one or more low-resolution (LR) images. Single image super-resolution (SISR) deals with one LR image while multi-frame super-resolution (MFSR) employs several LR ones to reach the HR output. MFSR pipeline consists of alignment, fusion, and reconstruction. We conduct a theoretical analysis using sparse coding (SC) and iterative shrinkage-thresholding algorithm to fill the gap of mathematical justification in the execution order of the optimal MFSR pipeline. Our analysis recommends executing alignment and fusion before the reconstruction stage (whether through deconvolution or SISR techniques). The suggested order ensures enhanced performance in terms of peak signal-to-noise ratio and structural similarity index. The optimal pipeline also reduces computational complexity compared to intuitive approaches that apply SISR to each input LR image. Also, we demonstrate the usefulness of SC in analysis of computer vision tasks such as MFSR, leveraging the sparsity assumption in natural images. Simulation results support the findings of our theoretical analysis, both quantitatively and qualitatively.

DiffLense: a conditional diffusion model for super-resolution of gravitational lensing data

Abstract Gravitational lensing data is frequently collected at low resolution due to instrumental limitations and observing conditions. Machine learning-based super-resolution techniques offer a method to enhance the resolution of these images, enabling more precise measurements of lensing effects and a better understanding of the matter distribution in the lensing system. This enhancement can significantly improve our knowledge of the distribution of mass within the lensing galaxy and its environment, as well as the properties of the background source being lensed. Traditional super-resolution techniques typically learn a mapping function from lower-resolution to higher-resolution samples. However, these methods are often constrained by their dependence on optimizing a fixed distance function, which can result in the loss of intricate details crucial for astrophysical analysis. In this work, we introduce DiffLense, a novel super-resolution pipeline based on a conditional diffusion model specifically designed to enhance the resolution of gravitational lensing images obtained from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). Our approach adopts a generative model, leveraging the detailed structural information present in Hubble space telescope (HST) counterparts. The diffusion model, trained to generate HST data, is conditioned on HSC data pre-processed with denoising techniques and thresholding to significantly reduce noise and background interference. This process leads to a more distinct and less overlapping conditional distribution during the model’s training phase. We demonstrate that DiffLense outperforms existing state-of-the-art single-image super-resolution techniques, particularly in retaining the fine details necessary for astrophysical analyses.

Unsupervised deep learning method for single image super-resolution of the thick pinhole imaging system using deep image prior

Thick pinhole imaging system is widely used for diagnosing intense pulsed radiation sources. However, owing to the trade-off among spatial resolution, field of view (FOV) and signal-to-noise ratio (SNR), the imaging system normally falls short in achieving high-precision spatial diagnosis. In this paper, we propose an unsupervised deep learning method for single image super-resolution (SISR) of the thick pinhole imaging system. The point spread function (PSF) of the imaging system is obtained by analytical calculation and Monte Carlo simulation methods, and the mathematical model of the imaging system is established using a linear equation. To solve the ill-posed inverse problem, we adopt randomly initialized deep convolutional neural networks (DCNNs) as an image prior without pre-training, which is named deep image prior (DIP). The results demonstrate that, by utilizing the SISR technique to increase the number of pixels in reconstructed images, the proposed DIP algorithm can mitigate the spatial resolution degradation caused by an insufficient spatial sampling frequency of the camera. Compared with various classical algorithms, the proposed DIP algorithm exhibits superior capabilities in recovering high-frequency signals and suppressing ringing artifacts. Furthermore, the convergence and robustness of the proposed DIP algorithm under different random seeds and SNR conditions are also verified.

Improving Single-Image Super-Resolution with Dilated Attention

Single-image super-resolution (SISR) techniques have become a vital tool for improving image quality and clarity in the rapidly evolving field of digital imaging. Convolutional neural network (CNN) and transformer-based SISR techniques are very popular. However, CNN-based techniques are not suitable when capturing long-range dependencies, and transformer-based techniques suffer from computational complexity. To tackle these problems, this paper proposes a novel method called dilated attention-based single-image super-resolution (DAIR). It comprises three components: low-level feature extraction, multi-scale dilated transformer block (MDTB), and high-quality image reconstruction. A convolutional layer is used to extract the base features from low-resolution images, which lays the foundation for subsequent processing. Dilated attention is introduced to MDTB to enhance its ability to capture image features at different scales and ensure superior image details and structure recovery. After that, MDTB refines these features to extract multi-scale global attributes and effectively grasps images’ long-distance relationships and features across multiple scales. Finally, low-level features obtained from feature extraction and multi-scale global features obtained from MDTB are aggregated to reconstruct high-resolution images. The comparison with existing methods validates the efficacy of the proposed method and demonstrates its advantage in improving image resolution and quality.

Lightweight image super-resolution for IoT devices using deep residual feature distillation network

The 5th industrial revolution is characterized by an extensive interconnection of embedded devices, which offer a range of services, including the monitoring of their environments. Images captured from remote cameras require enhancements for effective analysis. Despite recent progress in single-image super-resolution techniques by yielding impressive results through deep convolutional neural networks, the complexity of these advanced models renders them impractical for use on miniaturized Internet of Things (IoT) devices, primarily due to their limited computational capabilities and memory constraints. Furthermore, the rapid evolution of IoT devices necessitates efficient image super-resolution techniques, while existing advanced methods, based on deep convolutional neural networks, are too resource-intensive for these devices, and this gap highlights the need for a more suitable solution. In this study, we introduce a lightweight, efficient super-resolution model specifically designed for IoT devices. This model incorporates a novel deep residual feature distillation block (DRFDB), which leverages a depthwise-separable convolution block (DCB) for effective feature extraction. The focus is on reducing computational and memory demands without compromising on image quality. The proposed DCB extracts coarse features from given input features as calculation units, using two operations, depthwise and pointwise convolutions. These two operations are able to significantly reduce the number of parameters and floating-point operations while maintaining a PSNR value higher than the 90% threshold. We modify the proposed DCB and introduce a multi-kernel depthwise-separable convolution block (MKDCB) to fine-tune the model. The experiments, conduct on various standard datasets such as DIV2K, Set5, Set14, Urban100, and Manga109 by demonstrating that our model significantly outperforms existing methods in terms of both image quality and computational efficiency. The model shows improved performance metrics like PSNR, while requiring fewer parameters and less memory usage, making it highly suitable for IoT applications. This study presents a breakthrough in super-resolution for IoT devices, balancing high-quality image reconstruction with the limited resources of these devices.

A Feasibility Study on Extension of Measurement Distance in Vision Sensor Using Super-Resolution for Dynamic Response Measurement.

The current civil infrastructure conditions can be assessed through the measurement of displacement using conventional contact-type sensors. To address the disadvantages of traditional sensors, vision-based sensor measurement systems have been derived in numerous studies and proven as an alternative to traditional sensors. Despite the benefits of the vision sensor, it is well known that the accuracy of the vision-based displacement measurement is largely dependent on the camera extrinsic or intrinsic parameters. In this study, the feasibility study of a deep learning-based single image super-resolution (SISR) technique in a vision-based sensor system is conducted to alleviate the low spatial resolution of image frames at long measurement distance ranges. Additionally, its robustness is evaluated using shaking table tests. As a result, it is confirmed that the SISR can reconstruct definite images of natural targets resulting in an extension of the measurement distance range. Additionally, it is determined that the SISR mitigates displacement measurement error in the vision sensor-based measurement system. Based on this fundamental study of SISR in the feature point-based measurement system, further analysis such as modal analysis, damage detection, and so forth should be continued in order to explore the functionality of SR images by applying low-resolution displacement measurement footage.

Spatial and Channel Aggregation Network for Lightweight Image Super-Resolution.

Advanced deep learning-based Single Image Super-Resolution (SISR) techniques are designed to restore high-frequency image details and enhance imaging resolution through the use of rapid and lightweight network architectures. Existing SISR methodologies face the challenge of striking a balance between performance and computational costs, which hinders the practical application of SISR methods. In response to this challenge, the present study introduces a lightweight network known as the Spatial and Channel Aggregation Network (SCAN), designed to excel in image super-resolution (SR) tasks. SCAN is the first SISR method to employ large-kernel convolutions combined with feature reduction operations. This design enables the network to focus more on challenging intermediate-level information extraction, leading to improved performance and efficiency of the network. Additionally, an innovative 9 × 9 large kernel convolution was introduced to further expand the receptive field. The proposed SCAN method outperforms state-of-the-art lightweight SISR methods on benchmark datasets with a 0.13 dB improvement in peak signal-to-noise ratio (PSNR) and a 0.0013 increase in structural similarity (SSIM). Moreover, on remote sensing datasets, SCAN achieves a 0.4 dB improvement in PSNR and a 0.0033 increase in SSIM.

Use of Generative Adversarial Network Algorithm in Super-Resolution Images to Increase the Quality of Digital Elevation Models Based on ALOS PALSAR Data

Digital elevation models are responsible for providing altimetric information on a surface to be mapped. While global models of low and medium spatial resolution are available open source by several space agencies, the high- resolution ones, which are utilized in scales 1:25,000 and larger, are scarce and expensive. Here we address this limitation by the utilization of deep learning algorithms coupled with Single Image Super-Resolution techniques in digital elevation models to obtain better spatial quality versions from lower resolution inputs. The development of a GAN-based (Generative Adversarial Network-based) methodology enables the improvement of the initial spatial resolution of low-resolution images. In the geospatial data context, for example, these algorithms can be used with digital elevation models and satellite images. The methodological approach uses a dataset with digital elevation models SRTM (Shuttle Radar Topography Mission) (30 meters of spatial resolution) and ALOS PALSAR (12.5 meters of spatial resolution), created with the objective of allowing the study to be carried out, promoting the emergence of new research groups in the area as well as enabling the comparison between the results obtained. It has been found that by increasing the number of iterations the performance of the generated model was improved and the quality of the generated image increased. Furthermore, the visual analysis of the generated image against the high- and low-resolution ones showed a great similarity between the first two.

Deep Learned Singular Residual Network for Super Resolution Reconstruction

Single image super resolution (SISR) techniques produce images of high resolution (HR) as output from input images of low resolution (LR). Motivated by the effectiveness of deep learning methods, we provide a framework based on deep learning to achieve super resolution (SR) by utilizing deep singular-residual neural network (DSRNN) in training phase. Residuals are obtained from the difference between HR and LR images to generate LR-residual example pairs. Singular value decomposition (SVD) is applied to each LR-residual image pair to decompose into subbands of low and high frequency components. Later, DSRNN is trained on these subbands through input and output channels by optimizing the weights and biases of the network. With fewer layers in DSRNN, the influence of exploding gradients is reduced. This speeds up the learning process and also improves accuracy by using skip connections. The trained DSRNN parameters yield residuals to recover the HR subbands in the testing phase. Experimental analysis shows that the proposed method results in superior performance to existing methods in terms of subjective quality. Extensive testing results on popular benchmark datasets such as set5, set14, and urban100 for a scaling factor of 4 show the effectiveness of the proposed method across different qualitative evaluation metrics.

Brain Tumor Classification Using Hybrid Single Image Super-Resolution Technique With ResNext101_32× 8d and VGG19 Pre-Trained Models

Brain Tumor Classification Using Hybrid Single Image Super-Resolution Technique With ResNext101_32× 8d and VGG19 Pre-Trained Models

A review on Single Image Super Resolution techniques using generative adversarial network

Single Image Super Resolution (SISR) is a process to obtain a high pixel density and refined details from a low resolution (LR) image to get upscaled and sharper high-resolution (HR) image. In last decade, SISR based on Convolutional Neural Networks (CNN) have achieved impressive results for generating super-resolved images up to the size of ×3. This technique focuses on minimizing L1/L2 loss between real HR image and generated HR image without considering the perceptual quality of images. To improve upon, SISR based on Generative Adversarial Network (GAN) has gained researchers attention for generating visually pleasing images with reasonably minimizing L1/L2 loss for ×4 size images. The basic idea of GAN is to train two networks simultaneously, a Generator and a Discriminator such that generator can produce the super-resolved image for a given input LR image by learning real HR image distribution. This paper presents an overview of GAN based SISR techniques for further research as there are a few surveys in this area. Different GAN models have been classified in terms of architecture, algorithms, and loss functions including their benefits and limitations. Lastly, the research gaps as well as some possible solutions for the existing methods have been discussed.

APPLICATION OF PRE-TRAINED REAL-WORLD SUPER RESOLUTION MODELS TO OPTICAL SATELLITE IMAGE

Abstract. The single image super-resolution (SISR) technique refers to improving the resolution over the original image. In recent years, we use deep learning-based convolutional neural networks to improve the spatial resolution of images more reasonably. To train such deep learning models, we use training samples consisting of the original HR images and LR images obtained by bicubic downsampling. However, this method of training data using downsampling has a negative impact when applying the trained model to real images. That is because the downsampling function that occurs in the real image is unknown, and the hypothetically created LR image does not represent the resolution degradation that can realistically occur. Therefore, SISR methods that use realistic degradation called real-world super-resolution (RWSR) have been proposed. In this paper, we investigate how such RWSR methods using realistic degradation affect the SISR performance of satellite images. The results of applying the trained model to optical satellite images show that the RWSR method is not the most effective way to handle optical satellite images when compared to the deep learning method without modeling the degradation. In particular, we showed that the effect of RWSR with not only upsampling but also noise and blur removal is significant in the visibility of optical satellite images. Moreover, pre-trained RWSR models can be an aid in visually deciphering ground objects.

ENHANCED SUPER RESOLUTION FOR REMOTE SENSING IMAGERIES

Abstract. Single image super resolution (SISR) technology has been attracted much attention from remote sensing community due to its proven potentials in remote sensing applications. Existing SISR techniques varying from conventional interpolation methods to different network architectures. Generative adversarial networks (GANs) are one of the latest network architectures proven a greater potential as a SISR method whereas least attention has been given by the remote sensing community. Several studies have already been carried out on this context. However, yet there is no generalized GAN based approach to super resolve remote sensing imageries. Therefore, this study investigated the potentials of enhanced super resolution generative adversarial (ESRGAN) model to super resolve very high to medium resolution images from high to coarse resolution images for remote sensing applications. Two models were trained and Worldview-3 (WV3) images used as for very high resolution images. Whereas, down sampled WV3 and Sentinel-2(S2) were used as low resolution counterparts. Model performances were qualitatively and quantitatively analysed using standard metrics such as PSNR, SSIM, UIQI, CC, SAM, SID. Evaluation results emphasised super resolved images were preserved the original quality of the satellite images to a greater extent while improving its ground resolution.

Closed-Loop Residual Attention Network for Single Image Super-Resolution

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution

Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR) . However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatio-temporal remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatio-temporal remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.

Single Image Joint Motion Deblurring and Super-Resolution Using the Multi-Scale Channel Attention Modules

During the last decade, deep convolutional neural networks have significantly advanced the single image super-resolution techniques reconstructing realistic textural and spatial details. In classical image super-resolution problems, it is assumed that the low-resolution image has a certain downsampling degradation. However, complicated image degradations are inevitable in real-world scenarios, and motion blur is a common type of image degradation due to camera or scene motion during the image capturing process. This work proposes a fully convolutional neural network to reconstruct high-resolution sharp images from the given motion blurry low-resolution images. The deblurring subnetwork is based on multi-stage progressive architecture, while the super-resolution subnetwork is designed using the multi-scale channel attention modules. A simple and effective training strategy is employed where a pre-trained frozen deblurring module is used to train the super-resolution module. The deblurring module is unfrozen in the last training phase. Experiments show that, unlike the other methods, the proposed method reconstructs relatively small structures and textural details while successfully removing the complex motion blur. The implementation code and the pre-trained model are publicly available at https://github.com/misakshoyan/joint-motion-deblur-and-sr.

Residual Triplet Attention Network for Single-Image Super-Resolution

Single-image super-resolution (SISR) techniques have been developed rapidly with the remarkable progress of convolutional neural networks (CNNs). The previous CNNs-based SISR techniques mainly focus on the network design while ignoring the interactions and interdependencies between different dimensions of the features in the middle layers, consequently hindering the powerful learning ability of CNNs. In order to address this problem effectively, a residual triplet attention network (RTAN) for efficient interactions of the feature information is proposed. Specifically, we develop an innovative multiple-nested residual group (MNRG) structure to improve the learning ability for extracting the high-frequency information and train a deeper and more stable network. Furthermore, we present a novel lightweight residual triplet attention module (RTAM) to obtain the cross-dimensional attention weights of the features. The RTAM combines two cross-dimensional interaction blocks (CDIBs) and one spatial attention block (SAB) base on the residual module. Therefore, the RTAM is not only capable of capturing the cross-dimensional interactions and interdependencies of the features, but also utilizing the spatial information of the features. The simulation results and analysis show the superiority of the proposed RTAN over the state-of-the-art SISR networks in terms of both evaluation metrics and visual results.

An efficient transfer learning-based Super-Resolution model for Medical Ultrasound Image

There are many imaging modalities for clinical diagnosis, but we heavily use ultrasound imaging; some of the ultrasound images are low resolution because of the body’s internal weakness. This paper presents a novel unsupervised super-resolution framework for ultrasound image enhancement without considering any large sets of training samples which is the major concern in many single image super-resolution techniques. Here, the most powerful nonlinear mapping is introduced within convolutional neural networks to restore potentially useful and prominent spatial information generated from the model sets considered for image enhancements. Here, two techniques are used: dilated convolution and residual learning to increase the convergence and reconstruction accuracy in terms of quality measures. The potential metrics of dilated convolution in extract spatial information and residual learning are used to narrow the convergence time by learning only the difference between the test input and distorted input image. By evaluating the proposed framework on real ultrasound image sets, the performance metrics are validated. The proposed model outperformed the other competitive image enhancement models in the USSR field.

Robust deep learning-based multi-image super-resolution using inpainting

Traditional super-resolution techniques are generally presented as optimization problems with variations in the choice of optimization methods and cost functions. Even for the overdetermined cases, the problem is ill-conditioned. The situation is worsened when considering underdetermined cases with unknown regions due to occlusions or lack of data. Deep learning-based methods have shown promise in solving a similar problem. One recent advancement has come in the form of partial convolutions, which were developed to perform infilling of holes in images. When used in an appropriate deep neural network, this particular variant of the convolutional filter has shown great promise in approximating missing spatial information. The method described is formulated as a two-stage process. Lower resolution images are first registered and placed on a high-resolution grid. The problem is then treated as an in-painting task where the missing regions are reconstructed using a deep neural network with partial convolutional filters. We compare our method against deep learning-based single image super-resolution methods and classical multi-image super-resolution techniques using two similarity metrics and show that our method is more robust to occlusions and errors in registration while also producing higher quality outputs.