Super-resolution Image Reconstruction Research Articles

CNCAN: Contrast and normal channel attention network for super-resolution image reconstruction of crops and weeds

Numerous studies have been performed to apply camera vision technologies in robot-based agriculture and smart farms. In particular, to obtain high accuracy, it is essential to procure high-resolution (HR) images, which requires a high-performance camera. However, due to high costs it is difficult to widely apply the camera in agricultural robots. To overcome this limitation, we propose contrast and normal channel attention network (CNCAN) for super-resolution reconstruction (SR), which is the first research for the accurate semantic segmentation of crops and weeds even with low-resolution (LR) images captured by low-cost and LR camera. Attention block and activation function that considers high frequency and contrast information of images are used in CNCAN, and the residual connection method is applied to improve the learning stability.As a result of experimenting with three open datasets, namely, Bonirob, rice seedling and weed, and crop/weed field image (CWFID) datasets, the mean intersection of union (MIOU) results of semantic segmentation for crops and weeds with SR images through CNCAN were 0.7685, 0.6346, and 0.6931 in the Bonirob, rice seedling and weed, and CWFID datasets, respectively, confirming higher accuracy than other state-of-the-art methods for SR.

Infrared image super-resolution reconstruction based on visible light image guidance and recursive fusion

Due to the limitations of hardware equipment, infrared images generally have problems such as low resolution, blurred details and poor visual quality during acquisition. Using visible light images to guide the super-resolution reconstruction of infrared images is an effective way to improve the resolution of infrared images. However, the imaging principles of visible light images and infrared images are different, resulting in differences in detail information between the two images, so problems such as blur and ghosting may occur during reconstruction. This paper proposes an infrared image super-resolution network based on visible light image guidance and recursive fusion. In this network, a flow Fourier residual module is designed, and modules of different depths are used to extract information of different frequencies in visible light images and infrared images, so that each module focuses on the appropriate frequency information. At the same time, a hybrid attention module is used to obtain detail information in multimodal images from channel and spatial perspectives, and fuse them in a complementary way, which helps to eliminate the generation of artifacts. On this basis, a global recursive fusion branch is designed to consider the correlation between multi-layer features, adaptively fuse multi-layer features, and generate clearer high-resolution infrared images. Experimental results show that compared with the comparative methods, this method performs better in objective evaluation indicators; in terms of subjective visual comparison, the images reconstructed by this method have clearer textures and fewer artifacts, and better object distinction in complex environments.

A Review of Satellite-Based CO2 Data Reconstruction Studies: Methodologies, Challenges, and Advances

Carbon dioxide is one of the most influential greenhouse gases affecting human life. CO2 data can be obtained through three methods: ground-based, airborne, and satellite-based observations. However, ground-based monitoring is typically composed of sparsely distributed stations, while airborne monitoring has limited coverage and spatial resolution; they cannot fully reflect the spatiotemporal distribution of CO2. Satellite remote sensing plays a crucial role in monitoring the global distribution of atmospheric CO2, offering high observation accuracy and wide coverage. However, satellite remote sensing still faces spatiotemporal constraints, such as interference from clouds (or aerosols) and limitations from satellite orbits, which can lead to significant data loss. Therefore, the reconstruction of satellite-based CO2 data becomes particularly important. This article summarizes methods for the reconstruction of satellite-based CO2 data, including interpolation, data fusion, and super-resolution reconstruction techniques, and their advantages and disadvantages, it also provides a comprehensive overview of the classification and applications of super-resolution reconstruction techniques. Finally, the article offers future perspectives, suggesting that ideas like image super-resolution reconstruction represent the future trend in the field of satellite-based CO2 data reconstruction.

Image Super-Resolution Reconstruction Based on Dense Residual Attention and Multi-Scale Feature Fusion

In order to obtain super-resolution images with richer details and clearer textures, a method for image super-resolution reconstruction using dense residual attention and multi-scale fusion is proposed. First, different scale convolutions are used to fully extract shallow features of the image; then high-frequency features of the image are extracted through one three-layer cascaded multi-scale feature fusion and dense residual attention module, and the reuse of feature map is achieved; finally, residual branches are used to introduce shallow features and high-frequency features of each channel image, and the high-resolution images are reconstructed through up-sampling and sub-pixel convolution. The test results on the Set5, Set14, Bsd100, and Urban100 datasets show that the PSNR and SSIM of our model are superior to most current algorithms, especially in the case of [Formula: see text] reconstruction results. PSNR has improved by 0.2 dB on the Set5 and Bsd100 datasets, and the algorithm has a better subjective visual effect.

OGRN: optical-guided residual dense network for SAR image super-resolution reconstruction network

ABSTRACT Although Convolutional Neural Networks have significantly improved the development of SAR image super-resolution (SR) technology in recent years, it is a very challenging problem to reconstruct SAR image with large-scale factors, such as ×4 and ×8 due to limited available information from low-resolution image. The co-registered high-resolution optical image has been successfully applied to enhance the quality of SAR image due to its discriminative characteristics. Compared with single-frame SAR image SR reconstruction technology, optical image-guided SAR image SR reconstruction technology has better performance. This paper proposes an optical-guided residual dense network (OGRN) for SAR image super-resolution reconstruction network. The network is an end-to-end network. Its most important characteristic is that the SR model of SAR images can be obtained through optical image assistance during training, and high-quality SAR images can be generated without optical image assistance during testing. Firstly, a new dense residual backbone network that extracts high-frequency information is constructed. It can extract and propagate high-frequency features of SAR images efficiently to improve the feature representation ability of the network for high-frequency features. Then, the extracted high-frequency features of the SAR image are used to generate an optical image through the designed SAR-to-Optical image translation network. Finally, a reconstruction module is constructed based on the proposed fusion module for optical and SAR images. The fusion module fuses the high-frequency feature information of the extracted optical images with the SAR image features to provide features for SAR image SR reconstruction. Extensive experiments conducted on Sen1-2 and QXS datasets demonstrated that under the guidance of optical image, our OGRN can achieve excellent performance in both quantitative assessment metrics and visual quality.

Light field images super-resolution method based on hybrid low-dimensional spatial–angular interaction feature and linear complementation epipolar feature

Light-field (LF) cameras record 4D information about the intensity and angle of light, but limited by the resolution of the sensor, LF images require a trade-off between spatial and angular resolution, which results in generally low spatial resolution for LF images. We work on spatial super-resolution reconstruction (SR) of LF images, but due to the high dimensionality of 4D LF, it is difficult to process it directly, so we decompose 4D LF into low-dimensional sub-spaces. Since the 4D features of interest to different low-dimensional sub-spaces are different, we design a hybrid network structure—HSAESR, which processes multiple low-dimensional sub-spaces individually to adequately model the texture information in the 4D LF. HSAESR is designed to reconstruct Macro Pixel Images (MacPI) and Extremely Planar Images (EPI) respectively, and the final reconstruction results are weighted and fused. For MacPI, a CNN-based feature interaction module is designed using a novel asymptotic ’grid’ interaction approach to fully utilize the correlation between the 2D spatial dimension and the 2D angular dimension sub-spaces in 4D LF. For EPI, a Transformer-based epipolar feature linear complementation module is designed learning global and non-global multi-directional epipolar feature correlations, which learns feature linear correlation factor (α) and linear bias weight (β) through the correlation of non-global epipolar features, and corrects and complements the global epipolar correlation features through feature linear complementation. Extensive experiments are conducted on five datasets, the results show that HSAESR further improves the performance of LF images spatial SR, the reconstruction results are better than the comparison methods.

Vibration displacement measurement of bridge structural models using image super-resolution reconstruction and visual object detection network

Abstract Visual vibration measurement has emerged in the field of structural health monitoring in recent years, but it still has some shortcomings in terms of resolution, recognition rate and real-time performance. Considering the three aspects of recovering high-frequency image details, improving the compactness of the target bounding box, and reducing the computational time, we use the constructed image super-resolution reconstruction model and target detection model to measure the vibration displacement of the bridge structural model. First, we integrate the Transformer module into the Unet network with a simple structure. The Swin and Global Transformer Unet (SGTU) module constructed in this form can reduce the computational cost while reconstructing the large-resolution feature map target, and it can sharply edge information of the vibration target. We use the framework of the YOLOv5 algorithm as the backbone, and use the GhostBottleneck (GB) module to reduce the time for convolution operations to generate similar features. In addition, the proposed DWCBottleneck (DWCB) fusion module is also able to achieve high-level semantic fusion and network depth expansion with minimal computational cost. Finally, the center point offset of the bounding box predicted by the model can be used to obtain the displacement offset of the object in the image sequence. The position information of the target in the first frame image is used as the reference frame for calculating the offset, and the vibration displacement of the flexible structure in the image coordinate system is obtained by calculating the deviation of the displacement between the remaining frames and the first frame. We perform qualitative and quantitative comparisons in three aspects: video super-resolution reconstruction, visual detection robustness, and sensor vibration measurement displacement using a homemade vibration image dataset. The time-frequency domain displacement curves regressed by the visual vibration measurement algorithm are compared with the curves acquired after accelerometer acquisition, indicating the necessity of super-resolution reconstruction in visual vibration measurement.

Super-resolution deep learning image reconstruction: image quality and myocardial homogeneity in coronary computed tomography angiography

BackgroundThe recently introduced super-resolution (SR) deep learning image reconstruction (DLR) is potentially effective in reducing noise level and enhancing the spatial resolution. We aimed to investigate whether SR-DLR has advantages in the overall image quality and intensity homogeneity on coronary computed tomography (CT) angiography with four different approaches: filtered-back projection (FBP), hybrid iterative reconstruction (IR), DLR, and SR-DLR.MethodsSixty-three patients (mean age, 61 ± 11 years; range, 18–81 years; 40 men) who had undergone coronary CT angiography between June and October 2022 were retrospectively included. Image noise, signal to noise ratio, and contrast to noise ratio were quantified in both proximal and distal segments of the major coronary arteries. The left ventricle myocardium contrast homogeneity was analyzed. Two independent reviewers scored overall image quality, image noise, image sharpness, and myocardial homogeneity.ResultsImage noise in Hounsfield units (HU) was significantly lower (P < 0.001) for the SR-DLR (11.2 ± 2.0 HU) compared to those associated with other image reconstruction methods including FBP (30.5 ± 10.5 HU), hybrid IR (20.0 ± 5.4 HU), and DLR (14.2 ± 2.5 HU) in both proximal and distal segments. SR-DLR significantly improved signal to noise ratio and contrast to noise ratio in both the proximal and distal segments of the major coronary arteries. No significant difference was observed in the myocardial CT attenuation with SR-DLR among different segments of the left ventricle myocardium (P = 0.345). Conversely, FBP and hybrid IR resulted in inhomogeneous myocardial CT attenuation (P < 0.001). Two reviewers graded subjective image quality with SR-DLR higher than other image reconstruction techniques (P < 0.001).ConclusionsSR-DLR improved image quality, demonstrated clearer delineation of distal segments of coronary arteries, and was seemingly accurate for quantifying CT attenuation in the myocardium.

Accurate Inspection and Super-Resolution Reconstruction for Additive Manufactured Defects Based on Stokes Vector Method and Deep Learning

Defects in additive manufacturing processes are closely related to the mechanical and physical properties of the components. However, the extreme conditions of high temperatures, intense light, and powder during the manufacturing process present significant challenges for defect detection. Additionally, the high reflectivity of metallic components can cause pixels in image sensors to become overexposed, resulting in the loss of many defect signals. Thus, this paper mainly focuses on proposing an accurate inspection and super-resolution reconstruction method for additive manufactured defects based on Stokes vector and deep learning, where the Stokes vectors, polarization degree, and polarization angles of the inspected defects are effectively utilized to suppress the high reflectivity of metallic surfaces, enhance the contrast of defect regions, and highlight the boundaries of defects. Furthermore, a modified SRGAN model designated SRGAN-H is presented by employing an additional convolutional layer and activation functions, including Harswish and Tanh, to accelerate the convergence of the SRGAN-H network and improve the reconstruction of the additive manufactured defect region. The experiment results demonstrated that the SRGAN-H model outperformed SRGAN and traditional SR reconstruction algorithms in terms of the images of Stokes vectors, polarization degree, and polarization angles. For the scratch and hole test sets, the PSNR values were 33.405 and 31.159, respectively, and the SSIM values were 0.890 and 0.896, respectively. These results reflect the effectiveness of the SRGAN-H model in super-resolution reconstruction of scratch and hole images. For the scratch and hole images chosen in this study, the PSNR values of SRGAN-H for single image super-resolution reconstruction ranged from 31.86786 to 43.82374, higher than the results obtained by the pre-improvement SRGAN algorithm.

Image super-resolution reconstruction of vast-receptive-field pixel attention for precision measurement

Abstract Compared with traditional contact precision measurement, vision-based non-contact precision measurement has the features of low cost and flexible multi-point information extraction, but how to ensure the measurement accuracy of vision-based non-contact precision measurement is an urgent problem. Traditional thinking often focuses on hardware upgrades to improve image resolution, but this brings high costs and is limited by the physical characteristics of the hardware itself. In this paper, we start from the software aspect to improve the image resolution by using the super-resolution reconstruction algorithm and propose an image super-resolution reconstruction algorithm—Swin Transformer with a Vast-receptive-field Pixel Attention, which combines the vast-receptive-field pixel attention mechanism with the Swin Transformer self-attention mechanism, focuses on the learning of the high-frequency information features of the image. Experiments are conducted both in public datasets and real measurement images. Extensive experimental validation shows that the model can obtain more edge and high-frequency detail features in public datasets, and the objective evaluation index on Set5, Set14, B100, Urban100, and Manga109 datasets is improved by 0.06 dB on average compared with the existing algorithms. In actual measurements, the algorithm in this paper for USAF1951 resolution tablet, image super-resolution reconstruction image in the horizontal and vertical direction of the measurement accuracy increased by an average of 6.97%, the horizontal and vertical direction of the relative measurement accuracy of an average of 30.20% improvement. This study provides a potential development direction for vision-based non-contact precision measurement.

Super-resolution reconstruction of hydrate-bearing CT images for microscopic detection of pore

The pore structure of marine natural gas-hydrate-bearing sediments is a key factor related to the physical properties of reservoirs. However, the resolution of micro-computed tomography (micro-CT) images is unsuitable for the analysis of pore structures in fine-grained sediments. In this regard, super-resolution (SR) reconstruction technology is expected to improve the spatial resolution of micro-CT images. We present a self-supervised learning method that does not require high-resolution datasets as input images to complete the training and reconstruction processes. This method is an end-to-end network consisting of two subnetworks: an SR network and a downscaling network. We trained on a self-built dataset of hydrate samples from three different particle sizes. Compared with typical methods, the SR results indicate that our method provides high resolution while improving clarity. In addition, it has the highest consistency with the liquid saturation method with the subsequent calculation of porosity parameters. This study contributes to the investigation of seepage and energy transfer in sediments containing natural gas hydrates, which is particularly important for the exploration and development of marine natural gas hydrate resources.

A novel single image super-resolution reconstruction model based on edge-enhanced Siamese generative adversarial networks

Abstract Currently, most image super-resolution (SR) reconstruction algorithms are supervised, improving the overall visual quality of images based on a large amount of paired low-resolution (LR) and high-resolution (HR) image. However, collecting a large number of paired datasets is challenging, and the quality of the dataset can influence the reconstruction results. Although unsupervised methods have gained significant performance based on synthetic datasets, their effectiveness is relatively poor when applied to real-world or natural data. Focusing on those aforementioned issues, a novel image SR reconstruction algorithm of Edge-enhanced Siamese Generative Adversarial Network (EeSiGAN) is proposed. EeSiGAN belongs to the unsupervised category and does not require the involvement of HR images. Firstly, employing the Siamese GAN as the overall framework ensures the continuous consistency of image feature maps through the collaborative effect of support and main networks. Secondly, fusing the results of the two branches by using the Multi- scale Channel Attention Feature Fusion (MsCAFF) module. In addition, an Edge- enhanced Feature Distillation Block (EFDB) is introduced to enhance edge information in images and optimize the capability in feature extraction of network. Finally, Directional Variance Attention (DiVA) mechanism is used to obtain long-range spatial dependencies and simultaneously utilize inter-channel dependencies to achieve a more discriminative representation, thereby enhancing the overall quality of the recovered images. Extensive experimental results on synthetic and real datasets demonstrate that compared to other state-of-the-art unsupervised SR networks, the proposed EeSiGAN algorithm exhibits better performance in reconstructing images, producing clearer details and textures in the enlarged images.

Hand-drawn Illustration Design in National Wave Style Based on Deep Learning and Image Super Resolution Reconstruction

Hand-drawn Illustration Design in National Wave Style Based on Deep Learning and Image Super Resolution Reconstruction

Lightweight image super-resolution reconstruction based on mixed attention and global inductive bias network

Lightweight image super-resolution reconstruction based on mixed attention and global inductive bias network

A Super-Resolution and 3D Reconstruction Method Based on OmDF Endoscopic Images.

In the field of endoscopic imaging, challenges such as low resolution, complex textures, and blurred edges often degrade the quality of 3D reconstructed models. To address these issues, this study introduces an innovative endoscopic image super-resolution and 3D reconstruction technique named Omni-Directional Focus and Scale Resolution (OmDF-SR). This method integrates an Omnidirectional Self-Attention (OSA) mechanism, an Omnidirectional Scale Aggregation Group (OSAG), a Dual-stream Adaptive Focus Mechanism (DAFM), and a Dynamic Edge Adjustment Framework (DEAF) to enhance the accuracy and efficiency of super-resolution processing. Additionally, it employs Structure from Motion (SfM) and Multi-View Stereo (MVS) technologies to achieve high-precision medical 3D models. Experimental results indicate significant improvements in image processing with a PSNR of 38.2902 dB and an SSIM of 0.9746 at a magnification factor of ×2, and a PSNR of 32.1723 dB and an SSIM of 0.9489 at ×4. Furthermore, the method excels in reconstructing detailed 3D models, enhancing point cloud density, mesh quality, and texture mapping richness, thus providing substantial support for clinical diagnosis and surgical planning.

Infrared Image Super-Resolution Network Utilizing the Enhanced Transformer and U-Net.

Infrared images hold significant value in applications such as remote sensing and fire safety. However, infrared detectors often face the problem of high hardware costs, which limits their widespread use. Advancements in deep learning have spurred innovative approaches to image super-resolution (SR), but comparatively few efforts have been dedicated to the exploration of infrared images. To address this, we design the Residual Swin Transformer and Average Pooling Block (RSTAB) and propose the SwinAIR, which can effectively extract and fuse the diverse frequency features in infrared images and achieve superior SR reconstruction performance. By further integrating SwinAIR with U-Net, we propose the SwinAIR-GAN for real infrared image SR reconstruction. SwinAIR-GAN extends the degradation space to better simulate the degradation process of real infrared images. Additionally, it incorporates spectral normalization, dropout, and artifact discrimination loss to reduce the potential image artifacts. Qualitative and quantitative evaluations on various datasets confirm the effectiveness of our proposed method in reconstructing realistic textures and details of infrared images.

Super-Resolution Image Reconstruction of Wavefront Coding Imaging System Based on Deep Learning Network

Wavefront Coding (WFC) is an innovative technique aimed at extending the depth of focus (DOF) of optics imaging systems. In digital imaging systems, super-resolution digital reconstruction close to the diffraction limit of optical systems has always been a hot research topic. With the design of a point spread function (PSF) generated by a suitably phase mask, WFC could also be used in super-resolution image reconstruction. In this paper, we use a deep learning network combined with WFC as a general framework for images reconstruction, and verify its possibility and effectiveness. Considering the blur and additive noise simultaneously, we proposed three super-resolution image reconstruction procedures utilizing convolutional neural networks (CNN) based on mean square error (MSE) loss, conditional Generative Adversarial Networks (CGAN), and Swin Transformer Networks (SwinIR) based on mean absolute error (MAE) loss. We verified their effectiveness by simulation experiments. A comparison of experimental results shows that the SwinIR deep residual network structure based on MAE loss optimization criteria can generate more realistic super-resolution images with more details. In addition, we used a WFC camera to obtain a resolution test target and real scene images for experiments. Using the resolution test target, we demonstrated that the spatial resolution could be improved from 55.6 lp/mm to 124 lp/mm by the proposed super-resolution reconstruction procedure. The reconstruction results show that the proposed deep learning network model is superior to the traditional method in reconstructing high-frequency details and effectively suppressing noise, with the resolution approaching the diffraction limit.

A Remote Sensing Image Super-Resolution Reconstruction Model Combining Multiple Attention Mechanisms.

Remote sensing images are characterized by high complexity, significant scale variations, and abundant details, which present challenges for existing deep learning-based super-resolution reconstruction methods. These algorithms often exhibit limited convolutional receptive fields and thus struggle to establish global contextual information, which can lead to an inadequate utilization of both global and local details and limited generalization capabilities. To address these issues, this study introduces a novel multi-branch residual hybrid attention block (MBRHAB). This innovative approach is part of a proposed super-resolution reconstruction model for remote sensing data, which incorporates various attention mechanisms to enhance performance. First, the model employs window-based multi-head self-attention to model long-range dependencies in images. A multi-branch convolution module (MBCM) is then constructed to enhance the convolutional receptive field for improved representation of global information. Convolutional attention is subsequently combined across channels and spatial dimensions to strengthen associations between different features and areas containing crucial details, thereby augmenting local semantic information. Finally, the model adopts a parallel design to enhance computational efficiency. Generalization performance was assessed using a cross-dataset approach involving two training datasets (NWPU-RESISC45 and PatternNet) and a third test dataset (UCMerced-LandUse). Experimental results confirmed that the proposed method surpassed the existing super-resolution algorithms, including Bicubic interpolation, SRCNN, ESRGAN, Real-ESRGAN, IRN, and DSSR in the metrics of PSNR and SSIM across various magnifications scales.

Edge-enhanced infrared image super-resolution reconstruction model under transformer

Infrared images have important applications in military, security and surveillance fields. However, limited by technical factors, the resolution of infrared images is generally low, which seriously limits the application and development of infrared images in various fields. To address the problem of difficult recovery of edge information and easy ringing effect in the super-resolution reconstruction process of infrared images, an edge-enhanced infrared image super-resolution reconstruction model TESR under transformer is proposed. The main structure of this model is transformer. First, in view of the problem of difficult recovery of edge information of infrared images, an edge detection auxiliary network is designed, which can obtain more accurate edge information from the input low-resolution images and enhance the edge details during image reconstruction; then, the CSWin Transformer is introduced to compute the self-attention of horizontal and vertical stripes in parallel, so as to increase the receptive field of the model and enable it to utilize features with higher semantic levels. The super-resolution reconstruction model proposed in this paper can extract more comprehensive image information, and at the same time, it can obtain more accurate edge information to enhance the texture details of super-resolution images, and achieve better reconstruction results.

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism