Super-resolution Method Research Articles

Comparison of different algorithms to improve the quality of Image

Super-resolution (SR) become the most important task in image processing because of requirement of high quality of images. It becomes popular in technology like computer vision and pattern recognition. In Single Image Super-resolution (SR) uses the sequences of single low resolution (LR) images from high resolution (HR) to improve the quality of image. The SR is an image processing problem which aims to generate a high resolution (HR) image from low resolution input image. The SR is difficult because of the missing information in the given LR image. Thus in SR it is very important to boost the reconstruction performance so that, there is need of effective prior knowledge. The prior knowledge of image will solve the problem to improve the quality of generated image. The main aim of SR is to generate the HR image with good visual perception as similar as original image. The paper compare two methods of SR that is bi-linear method, and direction group sparsity of image method that is proposed one. These methods will evaluate the accuracy, subjective visual effect, quality of image, time complexity and PSNR, etc. These calculations will enhance nature of picture like better visual impact quality, lower reconstruction error and adequate calculation proficiency, and so on

Tuning the excitation laser power in a stochastic optical reconstruction microscope for Alexa Fluor 647 dye in Vectashield mounting media.

Super-resolution imaging techniques have fundamentally changed our understanding of cellular architecture and dynamics by surpassing the diffraction limit and enabling the visualization of subcellular details. The popular super-resolution method known as stochastic optical reconstruction microscopy (STORM) relies on the exact localization of single fluorescent molecules. The significance of employing Vectashield as a mounting medium for the super-resolution imaging scheme called direct STORM has recently been explored. Alexa Fluor 647 (AF647), one of the most popular dyes, shows significant blinking in Vectashield. However, to observe prominent blinking of the fluorophore for the reconstruction of super-resolved images, the power of the excitation laser needs to be tuned. This work demonstrates the tuning of excitation power density in the sample plane for superior imaging performance using AF647 in Vectashield. Samples comprising MDA-MB-231 breast cancer cell line are used for the experiments. The actin filaments of the cell are stained with phalloidin-conjugated AF647 dye. For the experiment, we employ a low-cost openFrame-based STORM system equipped with a programmable Arduino-regulated laser source emitting at 638nm. An excitation power density of 0.60 kW/cm2 at 638nm in the sample plane is observed to maximize the signal-to-noise ratio, the number of switching events, and the number of photons detected per event during image acquisition, thereby leading to the best imaging performance in terms of resolution. The outcome of this work will promote further STORM-based super-resolved imaging applications in cell biology using Alexa Fluor 647 in Vectashield.

Area Restoration of Channel Impulse Response With Time Decomposition Based Super-Resolution Method

Area Restoration of Channel Impulse Response With Time Decomposition Based Super-Resolution Method

MambaSR: Arbitrary-Scale Super-Resolution Integrating Mamba with Fast Fourier Convolution Blocks

Traditional single image super-resolution (SISR) methods, which focus on integer scale super-resolution, often require separate training for each scale factor, leading to increased computational resource consumption. In this paper, we propose MambaSR, a novel arbitrary-scale super-resolution approach integrating Mamba with Fast Fourier Convolution Blocks. MambaSR leverages the strengths of the Mamba state-space model to extract long-range dependencies. In addition, Fast Fourier Convolution Blocks are proposed to capture the global information in the frequency domain. The experimental results demonstrate that MambaSR achieves superior performance compared to different methods across various benchmark datasets. Specifically, on the Urban100 dataset, MambaSR outperforms MetaSR by 0.93 dB in PSNR and 0.0203 dB in SSIM, and on the Manga109 dataset, it achieves an average PSNR improvement of 1.00 dB and an SSIM improvement of 0.0093 dB. These results highlight the efficacy of MambaSR in enhancing image quality for arbitrary-scale super-resolution.

Infrared remote-sensing image super-resolution based on physical characteristic deduction

In this paper, we propose a high-confidence, super-resolution method for short-wave infrared remote-sensing images based on characteristic deduction form high-resolution visible images. First, based on the radiative transmission mechanism of full remote-sensing imaging link, a ground object characterization model based on the dark object method was established to achieve the inversion of the object’s ground reflection characteristics of high-resolution visible images. Second, the reflectance information of the specified infrared band can be deduced combined with the reflectance matching of the typical ground object spectral library. On this basis, combined with the characterization modeling of remote-sensing imaging link—such as atmosphere, optical system, platform, and detector—a high-resolution infrared remote-sensing image reconstruction method is proposed. Based on the imaging degradation model between high- and low-resolution infrared images, the results of high-resolution reconstruction were corrected using measured infrared images. Lastly, the final fusion-super-resolution image can be obtained. The results show that the proposed fusion-super-resolution method yields high-resolution infrared images more realistically than traditional image fusion algorithms and shows better structural similarity index and visual information fidelity results, verifying the effectiveness of the proposed infrared remote-sensing image fusion-super-resolution method.

A Super-Resolution and 3D Reconstruction Method Based on OmDF Endoscopic Images.

In the field of endoscopic imaging, challenges such as low resolution, complex textures, and blurred edges often degrade the quality of 3D reconstructed models. To address these issues, this study introduces an innovative endoscopic image super-resolution and 3D reconstruction technique named Omni-Directional Focus and Scale Resolution (OmDF-SR). This method integrates an Omnidirectional Self-Attention (OSA) mechanism, an Omnidirectional Scale Aggregation Group (OSAG), a Dual-stream Adaptive Focus Mechanism (DAFM), and a Dynamic Edge Adjustment Framework (DEAF) to enhance the accuracy and efficiency of super-resolution processing. Additionally, it employs Structure from Motion (SfM) and Multi-View Stereo (MVS) technologies to achieve high-precision medical 3D models. Experimental results indicate significant improvements in image processing with a PSNR of 38.2902 dB and an SSIM of 0.9746 at a magnification factor of ×2, and a PSNR of 32.1723 dB and an SSIM of 0.9489 at ×4. Furthermore, the method excels in reconstructing detailed 3D models, enhancing point cloud density, mesh quality, and texture mapping richness, thus providing substantial support for clinical diagnosis and surgical planning.

Dual path features interaction network for efficient image super-resolution

Image super-resolution (SR) is a crucial task in computer vision that involves reconstructing a low-resolution (LR) image into its high-resolution (HR) counterpart. Transformer-based methods excel at establishing long-range dependency but face challenges with high-complexity computations and capturing fine-grained features. Conversely, CNN-based methods are advantageous in capturing fine-grained features but are limited by fixed receptive fields. Additionally, most SR methods suffer from channel redundancy, leading to higher computational overhead. In this paper, we propose a Dual Path Features Interaction Network (DPFINet) to achieve efficient image SR, which consists of two components: a) To alleviate the issue of feature channel redundancy, a Local–Global Features Modeling (LGFM) method is newly proposed, which concurrently models global and local features by splitting features along different channels. In LGFM, a Shift Window Linear Attention (SWLA) layer is adopted to effectively capture global information through a large shift window based on the split features. Meanwhile, a Multi-Scale Detail Enhancement (MSDE) layer is designed, where the split other features are encoded to facilitate detail reconstruction through an interactive fusion of semantic and local information, thereby addressing the limitations of SWLA in capturing fine-grained features. b) A Cross-Level Features Interaction (CLFI) method is proposed to fuse global and local features modeled by different network structures (SWLA and MSDE), where a novel residual fusion mechanism is designed to preserve both global and local information while complementing each other. Extensive experiments demonstrate that our method outperforms most state-of-the-art SR methods on five benchmark datasets. Notably, during inference, our approach improves performance by 0.41 dB and reduces memory consumption by approximately 79% compared to DiVANet (Behjati et al., 2023) on the Manga109 (×4) dataset.

A Remote Sensing Image Super-Resolution Reconstruction Model Combining Multiple Attention Mechanisms.

Remote sensing images are characterized by high complexity, significant scale variations, and abundant details, which present challenges for existing deep learning-based super-resolution reconstruction methods. These algorithms often exhibit limited convolutional receptive fields and thus struggle to establish global contextual information, which can lead to an inadequate utilization of both global and local details and limited generalization capabilities. To address these issues, this study introduces a novel multi-branch residual hybrid attention block (MBRHAB). This innovative approach is part of a proposed super-resolution reconstruction model for remote sensing data, which incorporates various attention mechanisms to enhance performance. First, the model employs window-based multi-head self-attention to model long-range dependencies in images. A multi-branch convolution module (MBCM) is then constructed to enhance the convolutional receptive field for improved representation of global information. Convolutional attention is subsequently combined across channels and spatial dimensions to strengthen associations between different features and areas containing crucial details, thereby augmenting local semantic information. Finally, the model adopts a parallel design to enhance computational efficiency. Generalization performance was assessed using a cross-dataset approach involving two training datasets (NWPU-RESISC45 and PatternNet) and a third test dataset (UCMerced-LandUse). Experimental results confirmed that the proposed method surpassed the existing super-resolution algorithms, including Bicubic interpolation, SRCNN, ESRGAN, Real-ESRGAN, IRN, and DSSR in the metrics of PSNR and SSIM across various magnifications scales.

A Full Meshless Algorithm For Super-Resolution In Image Velocimetry Based On KNN-PTV And Constrained RBFs

"A novel full meshless super-resolution method for image velocimetry is proposed. This method builds upon the combination, presented in Tirelli et al. (2023a), of K-nearest neighbour Particle Tracking Velocimetry (Tirelli et al., 2023b, KNN-PTV) and constrained Radial Basis Functions regression (Sperotto et al., 2022, c-RBFs). The main novelty is that the algorithm is implemented here in a fully meshless version, i.e. it does not require the definition of a common Eulerian grid at any step of the process. KNN-PTV enhances the spatial resolution of vector fields by exploiting data coherence in space and time, even for non-time-resolved measurements. On the other and, c-RBFs provide an analytical representation of the field from scattered data while allowing the introduction of physical constraints. In this new version, the dictionary to blend data from different snapshots is computed with a meshless POD approach developed by the authors and presented in another contribution to this conference. This approach removes the last constraint of Eulerian grids and paves the way for a fully meshless algorithm. The algorithm is validated on a challenging 2D synthetic case, such as turbulent channel flow, and an experimental case involving the separation bubble around the frontal portion of a Ground Transportation System (GTS)."

Super-resolution reconstruction of background-oriented Schlieren displacement field based on the convolution neural network with the gradient loss function

Abstract To refine the displacement field of the background-oriented Schlieren method, a novel super-resolution method based on deep learning has been proposed and compared with the bicubic interpolation in this study. The gradient loss functions were first introduced into the hybrid downsampled skip-connection/multi-scale model to improve the reconstruction effect. The reconstruction effects of the new loss functions were compared with that of the traditional mean square error (MSE) loss function. The results show that the Laplace operator with average pooling exhibits better performance than the origin loss function in all the indexes including peak signal-to-noise ratio, MSE, MSE of the gradient, and the maximum MSE. In these four indexes, the MSE of the gradient and the maximum MSE performed especially better than the others, where the MSE of the gradient was reduced from 3. 0× 10−05 to 3.30 × 10−05, and the maximum MSE was reduced from 0.392 to 0.360.

Dual residual and large receptive field network for lightweight image super-resolution

In recent years, there has been rapid progress in single-image super-resolution (SISR) reconstruction technology based on deep learning, but many methods face challenges in practical application due to their excessive computational requirements. To address this issue, numerous lightweight super-resolution (SR) methods have been proposed. Convolutional neural networks (CNNs) have demonstrated outstanding performance in lightweight SR reconstruction tasks. However, existing CNN-based lightweight SR networks still suffer from redundancy due to the use of small convolutional kernels and deeper network architectures for feature extraction. To solve these problems, we propose a novel network called Dual Residual and Large Receptive Field Network (DRLN), which includes two main designs. Firstly, a Large Receptive Field Feature Extraction Block (LRFEB) is proposed, which concatenates large kernel depth separable convolutions and large kernel depth separable dilated convolutions, effectively capturing spatial contextual information without increasing the number of parameters or computational burden, and enhancing the overall performance of the model. Secondly, in the Efficient Feature Distillation Block (EFDB), we innovatively propose a dual residual structure, which greatly improve the learning ability and feature expression richness of the network, and significantly improve the reconstruction performance. Extensive experiments demonstrate that DRLN surpasses BSRN which is the champion of the model complexity track of the NTIRE 2022 Efficient SR Challenge on all benchmark datasets, with fewer parameters and less computation.

Self-Supervised Deep Blind Video Super-Resolution.

Existing deep learning-based video super-resolution (SR) methods usually depend on the supervised learning approach, where the training data is usually generated by the blurring operation with known or predefined kernels (e.g., Bicubic kernel) followed by a decimation operation. However, this does not hold for real applications as the degradation process is complex and cannot be approximated by these idea cases well. Moreover, obtaining high-resolution (HR) videos and the corresponding low-resolution (LR) ones in real-world scenarios is difficult. To overcome these problems, we propose a self-supervised learning method to solve the blind video SR problem, which simultaneously estimates blur kernels and HR videos from the LR videos. As directly using LR videos as supervision usually leads to trivial solutions, we develop a simple and effective method to generate auxiliary paired data from original LR videos according to the image formation of video SR, so that the networks can be better constrained by the generated paired data for both blur kernel estimation and latent HR video restoration. In addition, we introduce an optical flow estimation module to exploit the information from adjacent frames for HR video restoration. Experiments show that our method performs favorably against state-of-the-art ones on benchmarks and real-world videos.

Super-Resolution Method for Microwave Front-Looking Imaging Using Broadband Signals and Spatial Spectrum Shifting

Super-Resolution Method for Microwave Front-Looking Imaging Using Broadband Signals and Spatial Spectrum Shifting

Deep Residual-in-Residual Model-Based PET Image Super-Resolution with Motion Blur

Positron emission tomography (PET) is a non-invasive molecular imaging technique. The limited spatial resolution of PET images, due to technological and physical imaging constraints, directly affects the precise localization and interpretation of small lesions and biological processes. The super-resolution (SR) technique aims to enhance image quality by improving spatial resolution, thereby aiding clinicians in achieving more accurate diagnoses. However, most conventional SR methods rely on idealized degradation models and fail to effectively capture both low- and high-frequency information present in medical images. For the challenging SR reconstruction of PET images exhibiting motion-induced artefacts, a degradation model that better aligns with practical scanning scenarios was designed by us. Furthermore, we proposed a PET image SR method based on the deep residual-in-residual network (DRRN), focusing on the recovery of both low- and high-frequency information. By incorporating multi-level residual connections, our approach facilitates direct feature propagation across different network levels. This design effectively mitigates the lack of feature correlation between adjacent convolutional layers in deep networks. Our proposed method surpasses benchmark methods in both full-reference and no-reference metrics and subjective visual effects across small animal PET (SAPET), phantoms, and Alzheimer’s Disease Neuroimaging Initiative (ADNI) datasets. The experimental findings confirm the remarkable efficacy of DRRN in enhancing spatial resolution and mitigating blurring in PET images. In comparison to conventional SR techniques, this method demonstrates superior proficiency in restoring low-frequency structural texture information while simultaneously maintaining high-frequency details, thus showcasing exceptional multi-frequency information fusion capabilities.

A Novel Electrical Equipment Status Diagnosis Method Based on Super-Resolution Reconstruction and Logical Reasoning.

The accurate detection of electrical equipment states and faults is crucial for the reliable operation of such equipment and for maintaining the health of the overall power system. The state of power equipment can be effectively monitored through deep learning-based visual inspection methods, which provide essential information for diagnosing and predicting equipment failures. However, there are significant challenges: on the one hand, electrical equipment typically operates in complex environments, thus resulting in captured images that contain environmental noise, which significantly reduces the accuracy of state recognition based on visual perception. This, in turn, affects the comprehensiveness of the power system's situational awareness. On the other hand, visual perception is limited to obtaining the appearance characteristics of the equipment. The lack of logical reasoning makes it difficult for purely visual analysis to conduct a deeper analysis and diagnosis of the complex equipment state. Therefore, to address these two issues, we first designed an image super-resolution reconstruction method based on the Generative Adversarial Network (GAN) to filter environmental noise. Then, the pixel information is analyzed using a deep learning-based method to obtain the spatial feature of the equipment. Finally, by constructing the logic diagram for electrical equipment clusters, we propose an interpretable fault diagnosis method that integrates the spatial features and temporal states of the electrical equipment. To verify the effectiveness of the proposed algorithm, extensive experiments are conducted on six datasets. The results demonstrate that the proposed method can achieve high accuracy in diagnosing electrical equipment faults.

Restoration of Semantic-Based Super-Resolution Aerial Images

Currently, technologies for remote sensing image processing are actively developing, including both satellite images and aerial images obtained from video cameras of unmanned aerial vehicles. Often such images have artifacts such as low resolution, blurred image fragments, noise, etc. One way to overcome such limitations is to use modern technologies to restore super-resolution images based on deep learning methods. The specificity of aerial images is the presentation of texture and structural elements in a higher resolution than in satellite images, which objectively contributes to better results of restoration. The article provides a classification of super-resolution methods based on the main architectures of deep neural networks, namely convolutional neural networks, visual transformers and generative adversarial networks. The article proposes a method for reconstructing super-resolution aerial images SemESRGAN taking into account semantic features by using an additional deep network for semantic segmentation during the training stage. The total loss function, including adversarial losses, pixel-level losses, and perception losses (feature similarity), is minimized. Six annotated aerial and satellite image datasets CLCD, DOTA, LEVIR-CD, UAVid, AAD, and AID were used for the experiments. The results of image restoration using the proposed SemESRGAN method were compared with the basic architectures of convolutional neural networks, visual transformers and generative adversarial networks. Comparative results of image restoration were obtained using objective metrics PSNR and SSIM, which made it possible to evaluate the quality of restoration using various deep network models.

TDEGAN: A Texture-Detail-Enhanced Dense Generative Adversarial Network for Remote Sensing Image Super-Resolution

Image super-resolution (SR) technology can improve the resolution of images and provide clearer and more reliable remote sensing images of high quality to better serve the subsequent applications. However, when reconstructing high-frequency feature areas of remote sensing images, existing SR reconstruction methods are prone to artifacts that affect visual effects and make it difficult to generate real texture details. In order to address this issue, a texture-detail-enhanced dense generative adversarial network (TDEGAN) for remote sensing image SR is presented. The generator uses multi-level dense connections, residual connections, and Shuffle attention (SA) to improve the feature extraction ability. A PatchGAN-style discrimination network is designed to effectively perform local discrimination and helps the network generate rich, detailed features. To reduce the impact of artifacts, we introduce an artifact loss function, which is combined with the exponential moving average (EMA) technique to distinguish the artifacts generated from the actual texture details through local statistics, which can help the network reduce artifacts and generate more realistic texture details. Experiments show that TDEGAN can better restore the texture details of remote sensing images and achieves certain advantages in terms of evaluation indicators and visualization.

SIGAN: A Multi-Scale Generative Adversarial Network for Underwater Sonar Image Super-Resolution

Super-resolution (SR) is a technique that restores image details based on existing information, enhancing the resolution of images to prevent quality degradation. Despite significant achievements in deep-learning-based SR models, their application in underwater sonar scenarios is limited due to the lack of underwater sonar datasets and the difficulty in recovering texture details. To address these challenges, we propose a multi-scale generative adversarial network (SIGAN) for super-resolution reconstruction of underwater sonar images. The generator is built on a residual dense network (RDN), which extracts rich local features through densely connected convolutional layers. Additionally, a Convolutional Block Attention Module (CBAM) is incorporated to capture detailed texture information by focusing on different scales and channels. The discriminator employs a multi-scale discriminative structure, enhancing the detail perception of both generated and high-resolution (HR) images. Considering the increased noise in super-resolved sonar images, our loss function emphasizes the PSNR metric and incorporates the L2 loss function to improve the quality of the output images. Meanwhile, we constructed a dataset for side-scan sonar experiments (DNASI-I). We compared our method with the current state-of-the-art super-resolution image reconstruction methods on the public dataset KLSG-II and our self-built dataset DNASI-I. The experimental results show that at a scale factor of 4, the average PSNR value of our method was 3.5 higher than that of other methods, and the accuracy of target detection using the super-resolution reconstructed images can be improved to 91.4%. Through subjective qualitative comparison and objective quantitative analysis, we demonstrated the effectiveness and superiority of the proposed SIGAN in the super-resolution reconstruction of side-scan sonar images.

Analysis of Superresolution Methods in the Application to the Subsurface Sounding Radar for Detecting Hidden Objects

Adaptive methods have been developed for generating a radar image of a probed surface area. A block diagram of a subsurface sounding radar is described. The results of modeling and experimental studies are presented. Their use makes it possible to develop a subsurface sensing radar capable of effectively identifying hidden objects in concrete walls, as well as detecting cracks, voids and defects in the walls of buildings and structures, snow debris and avalanches, underground communications and organic bodies in the ground.

A Single Image High-Perception Super-Resolution Reconstruction Method Based on Multi-layer Feature Fusion Model with Adaptive Compression and Parameter Tuning

We propose a simple image high-perception super-resolution reconstruction method based on multi-layer feature fusion model with adaptive compression and parameter tuning. The aim is to further balance the high and low-frequency information of an image, enrich the detailed texture to improve perceptual quality, and improve the adaptive optimization and generalization of the model in the process of super-resolution reconstruction. First, an effective multi-layer fusion super-resolution (MFSR) basic model is constructed by the design of edge enhancement, refine layering, enhanced super-resolution generative adversarial network and other sub-models, and effective multi-layer fusion. This further enriches the image representation of features of different scales and depths and improves the feature representation of high and low-frequency information in a balanced way. Next, a total loss function of the generator is constructed with adaptive parameter tuning performance. The overall adaptability of the model is improved through adaptive weight distribution and fusion of content loss, perceptual loss, and adversarial loss, and improving the error while reducing the edge enhancement model. Finally, a fitness function with the evaluation perceptual function as the optimization strategy is constructed, and the model compression and adaptive tuning of MFSR are carried out based on the multi-mechanism fusion strategy. Consequently, the construction of the adaptive MFSR model is realized. Adaptive MFSR can maintain high peak signal to noise ratio and structural similarity on the test sets Set5, Set14, and BSD100, and achieve high-quality reconstructed images with low learned perceptual image patch similarity and perceptual index, while having good generalization capabilities.