Super-resolution Applications Research Articles

Superresolution of Lunar Satellite Images for Enhanced Robotic Traverse Planning: Maximizing the Value of Existing Data Products for Space Robotics

Lunar exploration missions require detailed and accurate planning to ensure their safety. Remote sensing data, such as optical satellite imagery acquired by lunar orbiters, are key for the identification of future landing and mission sites. Here robot- and astronaut-scale obstacles are the most relevant to resolve; however, the spatial resolution of the available image data is often insufficient, particularly in the poorly illuminated polar regions of the moon, leading to uncertainty. This work shows how a novel single-image superresolution (SISR) application, the Adversarial Network for Uncertainty-Based Image SR (ANUBIS), can enhance lunar surface imagery by improving the resolution by a factor of two, outperforming other approaches and benchmarks. The enhanced images improve the reliability and detail of lunar traverse planning and topographic reconstruction, while providing an estimate of the uncertainty associated with the enhancement process, vital to ensure mission planning integrity. This work demonstrates how machine-learning-driven processing can enhance existing data products to maximize their value for science and the exploration of the moon and other celestial bodies.

DAE-GAN: Underwater Image Super-Resolution Based on Symmetric Degradation Attention Enhanced Generative Adversarial Network

Underwater images often exhibit detail blurring and color distortion due to light scattering, impurities, and other influences, obscuring essential textures and details. This presents a challenge for existing super-resolution techniques in identifying and extracting effective features, making high-quality reconstruction difficult. This research aims to innovate underwater image super-resolution technology to tackle this challenge. Initially, an underwater image degradation model was created by integrating random subsampling, Gaussian blur, mixed noise, and suspended particle simulation to generate a highly realistic synthetic dataset, thereby training the network to adapt to various degradation factors. Subsequently, to enhance the network’s capability to extract key features, improvements were made based on the symmetrically structured blind super-resolution generative adversarial network (BSRGAN) model architecture. An attention mechanism based on energy functions was introduced within the generator to assess the importance of each pixel, and a weighted fusion strategy of adversarial loss, reconstruction loss, and perceptual loss was utilized to improve the quality of image reconstruction. Experimental results demonstrated that the proposed method achieved significant improvements in peak signal-to-noise ratio (PSNR) and underwater image quality measure (UIQM) by 0.85 dB and 0.19, respectively, significantly enhancing the visual perception quality and indicating its feasibility in super-resolution applications.

SuperVidConform: Conformation detail-preserving network (CDPN) for video super-resolution

Video Super Resolution (VSR) applications extensively utilize deep learning-based methods. Several VSR methods primarily focus on improving the fine-patterns within reconstructed video frames. It frequently overlooks the crucial aspect of keeping conformation details, particularly sharpness. Therefore, reconstructed video frames often fail to meet expectations. In this paper, we propose a Conformation Detail-Preserving Network (CDPN) named as SuperVidConform. It focuses on restoring local region features and maintaining the sharper details of video frames. The primary focus of this work is to generate the high-resolution (HR) frame from its corresponding low-resolution (LR). It consists of two parts: (i) The proposed model decomposes confirmation details from the ground-truth HR frames to provide additional information for the super-resolution process, and (ii) These video frames pass to the temporal modelling SR network to learn local region features by residual learning that connects the network intra-frame redundancies within video sequences. The proposed approach is designed and validated using VID4, SPMC, and UDM10 datasets. The experimental results show the proposed model presents an improvement of 0.43 dB (VID4), 0.78 dB (SPMC), and 0.84 dB (UDM10) in terms of PSNR. Further, the CDPN model set new standards for the performance of self-generated surveillance datasets.

Residual SwinV2 transformer coordinate attention network for image super resolution

Swin Transformers have been designed and used in various image super-resolution (SR) applications. One of the recent image restoration methods is RSTCANet, which combines Swin Transformer with Channel Attention. However, for some channels of images that may carry less useful information or noise, Channel Attention cannot automatically learn the insignificance of these channels. Instead, it tries to enhance their expression capability by adjusting the weights. It may lead to excessive focus on noise information while neglecting more essential features. In this paper, we propose a new image SR method, RSVTCANet, based on an extension of Swin2SR. Specifically, to effectively gather global information for the channel of images, we modify the Residual SwinV2 Transformer blocks in Swin2SR by introducing the coordinate attention for each two successive SwinV2 Transformer Layers (S2TL) and replacing Multi-head Self-Attention (MSA) with Efficient Multi-head Self-Attention version 2 (EMSAv2) to employ the resulting residual SwinV2 Transformer coordinate attention blocks (RSVTCABs) for feature extraction. Additionally, to improve the generalization of RSVTCANet during training, we apply an optimized RandAugment for data augmentation on the training dataset. Extensive experimental results show that RSVTCANet outperforms the recent image SR method regarding visual quality and measures such as PSNR and SSIM.

Self-supervised multi-magnification feature enhancement for segmentation of hepatocellular carcinoma region in pathological images

Accurate segmentation of hepatocellular carcinoma regions is a critical step in surgical evaluation of whole slide pathological images. Recently, emerging multi-magnification learning-based methods have shown promise with evaluation of whole slide images. However, traditional multi-magnification learning segmentation models focus on uniformity and fail to effectively exploit different magnification information. The purpose of this study is to develop a novel multi-magnification learning segmentation model by effectively utilizing diverse magnification information. Therefore, we developed a novel multi-magnification feature self-enhancement network to extract magnification-specific features from multi-magnification images via self-supervised learning without extra supervision. Specifically, the proposed network enhances feature information via a super-resolution module based on self-supervision to transfer the representative information into a segmentation encoder, which takes full advantage of the pyramid storage characteristic of whole slide images. Moreover, a multi-scale feature fusion module based on the attention mechanism is used to fuse pretrained multi-scale features through a gated unit block, which is designed to fuse features from generation task into and segmentation task. We evaluated this method using the hepatocellular carcinoma data set from The Cancer Genome Atlas with cross-validation and achieved a 0.829 Dice similarity coefficient, nearly a 3% improvement over state-of-the-art models used for segmentation. To the best of our knowledge, this is the first study of the application of super-resolution as a self-enhancement network for histopathological image segmentation via self-supervised pretext learning. The code will be available at https://github.com/SH-Diao123/Self-Supervised-Multi-magnification-Segmentation.

Exploring super-resolution spatial downscaling of several meteorological variables and potential applications for photovoltaic power

We applied a perfect prognosis approach to downscale four meteorological variables that affect photovoltaic (PV) power output using four machine learning (ML) algorithms. In addition to commonly investigated variables, such as air temperature and precipitation, we also focused on wind speed and surface solar radiation, which are not frequently examined. The downscaling performance of the four variables followed the order of: temperature > surface solar radiation > wind speed > precipitation. Having assessed the dependence of the downscaling accuracy on the scaling factor, we focused on a super-resolution downscaling. We found that the convolutional neural network (CNN) generally outperformed the other linear and non-linear algorithms. The CNN was further able to reproduce extremes. With the rapid transition from coal to renewables, the need to evaluate low solar output conditions at a regional scale is expected to benefit from CNNs. Because weather affects PV power output in multiple ways, and future climate change will modify meteorological conditions, we focused on obtaining exemplary super-resolution application by evaluating future changes in PV power outputs using climate simulations. Our results confirmed the reliability of the CNN method for producing super-resolution climate scenarios and will enable energy planners to anticipate the effects of future weather variability.

Application of super-resolution and ultrafast ultrasound to reveal the characteristics of vascular blood flow changes after rat spinal cord injury at different segments

Objective: To investigate the changes of spinal vascular blood flow in SD rats after cervical, thoracic and lumbar spinal cord injury (SCI) using super-resolution ultrafast ultrasound technology. Methods: A total of 9 SD rats were used to construct SCI models at different segments using a 50 g aneurysm clip. Super-resolution ultrafast ultrasound technology was used to perform vascular blood flow imaging on the spinal cord of rats before and after injury at 6 hours, obtaining quantitative information such as spinal cord vascular density and blood flow velocity. Results: Ultrasound imaging showed that after SCI, the vascular density in the thoracic segment decreased (18.16%±1.04%) more than in the cervical segment (11.42%±1.39%) and lumbar segment (13.88%±1.43%, both P<0.05). The length of the spinal cord with decreased vascular density in the thoracic segment [(4.80±0.34)mm] was longer than that in the cervical segment [(2.80±0.57)mm] and lumbar segment [(3.10±0.36)mm, both P<0.05]. After injury, the decrease of blood flow in the thoracic segment [(8.87±0.85)ml/min] was higher than that in the cervical segment [(4.88±0.56)ml/min] and lumbar segment [(6.19±0.71)ml/min, both P<0.05]. HE staining and Nissl staining showed that the proportion of cavity area after thoracic SCI (11.53%±0.93%) was higher than that in the cervical segment (4.90%±1.72%) and lumbar segment (7.64%±0.84%, both P<0.05). The number of Nissl bodies in the thoracic segment (18.0±5.3) was also lower than that in the cervical segment (32.3±5.1) and lumbar segment (37.0±5.6) (both P<0.05). Conclusions: There are different changes in vascular blood flow after SCI in different segments of rats. The same injury causes the most severe damage to blood vessels in the thoracic spinal cord, followed by the lumbar spinal cord, and the cervical spinal cord has the least damage.

Easy Access to Super-Resolution Applications with DeepSIM

Abstract Super-resolution microscopy is becoming more and more relevant and widespread for solving complex biological questions. However, many super-resolution techniques require high-level expertise and specific sample preparation protocols limiting the kind of fluorophore that can be used and consequently the applications. CrestOptics DeepSIM supports researchers deploying standard sample preparation protocols used for confocal microscopy while offering staggering resolution, the possibility of doing live cell imaging, and the ability to image deep into tissues. DeepSIM also allows staining with commonly used fluorescent markers, making it compatible with a large variety of samples and applications.

FDSR: An Interpretable Frequency Division Stepwise Process Based Single-Image Super-Resolution Network.

Deep learning has excelled in single-image super-resolution (SISR) applications, yet the lack of interpretability in most deep learning-based SR networks hinders their applicability, especially in fields like medical imaging that require transparent computation. To address these problems, we present an interpretable frequency division SR network that operates in the image frequency domain. It comprises a frequency division module and a step-wise reconstruction method, which divides the image into different frequencies and performs reconstruction accordingly. We develop a frequency division loss function to ensure that each reconstruction module (ReM) operates solely at one image frequency. These methods establish an interpretable framework for SR networks, visualizing the image reconstruction process and reducing the black box nature of SR networks. Additionally, we revisited the subpixel layer upsampling process by deriving its inverse process and designing a displacement generation module. This interpretable upsampling process incorporates subpixel information and is similar to pre-upsampling frameworks. Furthermore, we develop a new ReM based on interpretable Hessian attention to enhance network performance. Extensive experiments demonstrate that our network, without the frequency division loss, outperforms state-of-the-art methods qualitatively and quantitatively. The inclusion of the frequency division loss enhances the network's interpretability and robustness, and only slightly decreases the PSNR and SSIM metrics by an average of 0.48 dB and 0.0049, respectively.

Efficient multi-branch dynamic fusion network for super-resolution of industrial component image

This work aims to promote the application of a high-performance super-resolution (SR) method in industry. Considering the lack of industrial datasets to evaluate performance, an industrial image SR dataset called WCI110 is first established, comprising 110 typical welding component images with 2040 × 1524 pixels. Subsequently, a parallel fusion structure of CNN and Transformer (FPFCT) is designed to achieve a high-quality reconstruction of component images. This model mainly contains the flexible parallel fusion (FPF) block and dynamic edge attention (DEA) network. The irreparable information loss caused by the inheritable sampling imperfection of CNN and Transformer can be effectively avoided by the parallel fusion structure in FPF block, while the contour features of multi-scale targets can be dynamically enhanced by DEA modules. The results show that the performance of FPFCT is higher than most advanced SR methods based on CNN, Transformer, or hybrid of CNN and Transformer. The images reconstructed by FPFCT are shown to help find and locate defects more effectively than other methods because its reconstructed target is closer to ground truth in terms of contour sharpness and size. Moreover, FPFCT achieves a remarkable advance in model parameters and time consumption, with a drop of more than 37.5 % in model parameters compared to the state-of-the-art SwinIR, and a reduction of more than 48.9 % in single-image delay time during reconstruction. The high efficiency of FPFCT makes it a promising image preprocessing tool for industrial images.

Remote Sensing Image Super-Resolution Using Texture Enhancing Generative Adversarial Network

&lt;p&gt;Single image super-resolution (SISR) brings excellent improvement in remote sensing applications, which has been widely studied in recent years. A method named TFSRGAN of remote sensing single image super-resolution based on generative adversarial network is proposed in this paper to address the problems of poor reconstruction visual quality and smooth details in traditional algorithms. In the proposed framework, s dense residual connection method is proposed to fuse the deep features from each residual block based on the SRGAN network, and the channel attention mechanism is added into the residual block to combinate the channel information. In addition, the network employs an edge extractor to divide the low-resolution image into low-frequency image and high-frequency image as the input of generator to improve the effect of texture reconstruction. Extensive comparison experiments were performed using AID, UCAS_AOD and China Gaofen-1 datasets, the SR results demonstrate that the proposed TFSRGAN framework outperforms the state-of-the-art algorithms including VDSR, SRGAN and ESRGAN in terms of objective evaluation metrics and subjective visual perception. The ground targets detection experiments represent that the proposed TFSRGAN can significantly improve the effect in remote sensing super-resolution application.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

Real-Time Video Super-Resolution with Spatio-Temporal Modeling and Redundancy-Aware Inference.

Video super-resolution aims to generate high-resolution frames from low-resolution counterparts. It can be regarded as a specialized application of image super-resolution, serving various purposes, such as video display and surveillance. This paper proposes a novel method for real-time video super-resolution. It effectively exploits spatial information by utilizing the capabilities of an image super-resolution model and leverages the temporal information inherent in videos. Specifically, the method incorporates a pre-trained image super-resolution network as its foundational framework, allowing it to leverage existing expertise for super-resolution. A fast temporal information aggregation module is presented to further aggregate temporal cues across frames. By using deformable convolution to align features of neighboring frames, this module takes advantage of inter-frame dependency. In addition, it employs a hierarchical fast spatial offset feature extraction and a channel attention-based temporal fusion. A redundancy-aware inference algorithm is developed to reduce computational redundancy by reusing intermediate features, achieving real-time inferring speed. Extensive experiments on several benchmarks demonstrate that the proposed method can reconstruct satisfactory results with strong quantitative performance and visual qualities. The real-time inferring ability makes it suitable for real-world deployment.

Zero-Dimensional Carbon Nanomaterials for Fluorescent Sensing and Imaging.

Advances in nanotechnology and nanomaterials have attracted considerable interest and play key roles in scientific innovations in diverse fields. In particular, increased attention has been focused on carbon-based nanomaterials exhibiting diverse extended structures and unique properties. Among these materials, zero-dimensional structures, including fullerenes, carbon nano-onions, carbon nanodiamonds, and carbon dots, possess excellent bioaffinities and superior fluorescence properties that make these structures suitable for application to environmental and biological sensing, imaging, and therapeutics. This review provides a systematic overview of the classification and structural properties, design principles and preparation methods, and optical properties and sensing applications of zero-dimensional carbon nanomaterials. Recent interesting breakthroughs in the sensitive and selective sensing and imaging of heavy metal pollutants, hazardous substances, and bioactive molecules as well as applications in information encryption, super-resolution and photoacoustic imaging, and phototherapy and nanomedicine delivery are the main focus of this review. Finally, future challenges and prospects of these materials are highlighted and envisaged. This review presents a comprehensive basis and directions for designing, developing, and applying fascinating fluorescent sensors fabricated based on zero-dimensional carbon nanomaterials for specific requirements in numerous research fields.

Super-resolution in thin section of lacustrine shale reservoirs and its application in mineral and pore segmentation

Lacustrine shale reservoirs present intricate attributes such as the prevalence of lamination, rapid sedimentary phase transitions, and pronounced heterogeneity. These factors introduce substantial challenges in analyzing and comprehending reservoir characteristics. Thin-section imaging offers a direct medium to observe these traits, yet the intrinsic compromise between image resolution and field of view impedes the concurrent capture of comprehensive details and contextual overview. This study delves into the application of super-resolution (SR) techniques to augment the segmentation of thin-section images from lacustrine shale, an unconventional reservoir. SR application furnishes high-resolution images, facilitating a robust analysis of morphology, texture, edge properties, and target classification. Utilizing data from the lacustrine shale reservoir of the Ordos Basin, we evaluate our methodology and assess the impact of SR enhancement on segmentation. Quantitative results indicate significant improvements, with the Jaccard index for shale increasing from 0.4790 (Low-Resolution) to 0.7803 (ESRGAN) in the Y channel of the YCrCb color space after level set segmentation, exemplifying the efficacy of SR in shale gas and oil reservoirs. This research underscores the necessity to consider lacustrine shale's unique features while formulating and implementing SR techniques for improved information extraction. Furthermore, it highlights SR's potential for propelling future research and industry-specific applications.

Self-quenched Fluorophore Dimers for DNA-PAINT and STED Microscopy.

Super-resolution techniques like single-molecule localisation microscopy (SMLM) and stimulated emission depletion (STED) microscopy have been extended by the use of non-covalent, weak affinity-based transient labelling systems. DNA-based hybrid systems are a prominent example among these transient labelling systems, offering excellent opportunities for multi-target fluorescence imaging. However, these techniques suffer from higher background relative to covalently bound fluorophores, originating from unbound fluorophore-labelled single-stranded oligonucleotides. Here, we introduce short-distance self-quenching in fluorophore dimers as an efficient mechanism to reduce background fluorescence signal, while at the same time increasing the photon budget in the bound state by almost 2-fold. We characterise the optical and thermodynamic properties of fluorophore-dimer single-stranded DNA, and show super-resolution imaging applications with STED and SMLM with increased spatial resolution and reduced background.

Experimental verification of the super-resolution focusing optical needle and imaging with a new type Fresnel zone plate

This paper presents the experimental demonstration of the mixed Fresnel zone plate (M-FZP) for super-resolution focusing and imaging applications. The M-FZP, with a processing error of 5 nm, is fabricated using focused ion beam etching (FIB). The influence of FIB processing parameters and processing scanning methods on the quality of M-FZP is explored. Results show that the annular scanning processing method, performed from inside to outside, is more suitable for manufacturing M-FZPs. At a wavelength of λ = 633 nm, the planar M-FZP exhibits a focusing optical-needle with the smallest full width at half maximum of 295 nm (0.466 λ) and 283 nm (0.447 λ) in x and y directions, respectively. It also possesses a depth of focus of 3.9μm and a central focal length of 9.073μm. A device based on M-FZP is constructed in combination with confocal technology, and a corresponding experiment is conducted to enable super-resolution imaging. This new microstructure provides a new solution for super-resolution focusing and imaging.

Super-resolution analysis via machine learning: a survey for fluid flows

This paper surveys machine-learning-based super-resolution reconstruction for vortical flows. Super resolution aims to find the high-resolution flow fields from low-resolution data and is generally an approach used in image reconstruction. In addition to surveying a variety of recent super-resolution applications, we provide case studies of super-resolution analysis for an example of two-dimensional decaying isotropic turbulence. We demonstrate that physics-inspired model designs enable successful reconstruction of vortical flows from spatially limited measurements. We also discuss the challenges and outlooks of machine-learning-based super-resolution analysis for fluid flow applications. The insights gained from this study can be leveraged for super-resolution analysis of numerical and experimental flow data.Graphical abstract

Optimized instance segmentation by super-resolution and maximal clique generation

The rise of surveillance systems has led to exponential growth in collected data, enabling several advances in Deep Learning to exploit them and automate tasks for autonomous systems. Vehicle detection is a crucial task in the fields of Intelligent Vehicle Systems and Intelligent Transport systems, making it possible to control traffic density or detect accidents and potential risks. This paper presents an optimal meta-method that can be applied to any instant segmentation model, such as Mask R-CNN or YOLACT++. Using the initial detections obtained by these models and super-resolution, an optimized re-inference is performed, allowing the detection of elements not identified a priori and improving the quality of the rest of the detections. The direct application of super-resolution is limited because instance segmentation models process images according to a fixed dimension. Therefore, in cases where the super-resolved images exceed this fixed size, the model will rescale them again, thus losing the desired effect. The advantages of this meta-method lie mainly in the fact that it is not required to modify the model architecture or re-train it. Regardless of the size of the images given as input, super-resolved areas that fit the defined dimension of the object segmentation model will be generated. After applying our proposal, experiments show an improvement of up to 8.1% for the YOLACT++ model used in the Jena sequence of the CityScapes dataset.

Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

As the demand for high quality and high resolution in real-time rendering grows, superresolution is on its way to becoming a necessary component in modern real-time rendering applications (e.g., video games). The superresolution technique allows graphic applications to save computational costs by rendering at a lower resolution and reconstructing a high-resolution result. Nvidia introduced DLSS to the market as the first superresolution application in 2020, and NSRR was published on Siggraph the same year. Each of these approaches has shown powerful capabilities and is well suited to the needs of the industrial sector. In this paper, the authors propose the optimization potential of superresolution algorithms by introducing feature enhancement and feature caching modules and attempt to improve the current algorithms.

High-throughput single-molecule localization microscopy: Potential clinical applications.

High-throughput single-molecule localization microscopy: Potential clinical applications.