Super-resolution Research Articles

Improved diagnostic confidence using Super Resolution CEUS imaging in testicular lesions.

Ultrasound is the most used interdisciplinary non-ionizing imaging technique in clinical pathologies of the testis. The testis may be affected by a plethora of different disorders such as vasculopathies, trauma, infections and manifestations of primary and secondary malignant masses. Conventional ultrasound represents the basic imaging modality of choice to assess scrotal disorders. Contrast-enhanced ultrasound (CEUS) can provide further information to distinguish between benign and malignant testicular mass lesions. The recent introduction of Super Resolution CEUS Micro-Vascular Imaging (MVI SR) and Time of Arrival (TOA SR) parametric mapping compliments the information provided by conventional CEUS, since these two new post-processing techniques improve the visualization of microvascular structures with slow blood flow and provide high-resolution images of the peak contrast enhancement and temporal perfusion patterns. This paper gives a comprehensive overview of differential diagnoses of the testicular disorder and their corresponding sono-morphologic correlates based on representative cases of the Interdisciplinary Ultrasound Center of the University Hospital Munich.

Nanoporous Submicron Gold Particles Enable Nanoparticle-Based Localization Optoacoustic Tomography (nanoLOT).

Localization optoacoustic tomography (LOT) has recently emerged as a transformative super-resolution technique breaking through the acoustic diffraction limit in deep-tissue optoacoustic (OA) imaging via individual localization and tracking of particles in the bloodstream. However, strong light absorption in red blood cells has previously restricted per-particle OA detection to relatively large microparticles, ≈5µm in diameter. Herein, it is demonstrated that submicron-sized porous gold nanoparticles, ≈600nm in diameter, can be individually detected for noninvasive super-resolution imaging with LOT. Ultra-high-speed bright-field microscopy revealed that these nanoparticles generate microscopic plasmonic vapor bubbles, significantly enhancing opto-acoustic energy conversion through a nano-to-micro size transformation. Comprehensive in vitro and in vivo tests further demonstrated the biocompatibility and biosafety of the particles. By reducing the detectable particle size by an order of magnitude, nanoLOT enables microangiographic imaging with a significantly reduced risk of embolisms from particle aggregation and opens new avenues to visualize how nanoparticles reach vascular and potentially extravascular targets. The performance of nanoLOT for non-invasive imaging of microvascular networks in the murine brain anticipates new insights into neurovascular coupling mechanisms and longitudinal microcirculatory changes associated with neurodegenerative diseases.

Innovative sample preparation using alcohol dehydration and high refractive index medium enables acquisition of two-channel super-resolution 3D STED image of an entire oocyte.

Super-resolution (SR) microscopy is a cutting-edge method that can provide detailed structural information with high resolution. However, the thickness of the specimen has been a major limitation for SR methods, and large biological structures have posed a challenge. To overcome this, the key step is to optimise sample preparation to ensure optical homogeneity and clarity, which can enhance the capabilities of SR methods for the acquisition of thicker structures. Oocytes are the largest cells in the mammalian body and are crucial objects in reproductive biology. They are especially useful for studying membrane proteins. However, oocytes are extremely fragile and sensitive to mechanical manipulation and osmotic shocks, making sample preparation a critical and challenging step. We present an innovative, simple and sensitive approach to oocyte sample preparation for 3D STED acquisition. This involves alcohol dehydration and mounting into a high refractive index medium. This extended preparation procedure allowed us to successfully obtain a unique two-channel 3D STED SR image of an entire mouse oocyte. By optimising sample preparation, it is possible to overcome current limitations of SR methods and obtain high-resolution images of large biological structures, such as oocytes, in order to study fundamental biological processes. Lay Abstract: Super-resolution (SR) microscopy is a cutting-edge tool that allows scientists to view incredibly fine details in biological samples. However, it struggles with larger, thicker specimens, as they need to be optically clear and uniform for the best imaging results. In this study, we refined the sample preparation process to make it more suitable for SR microscopy. Our method includes carefully dehydrating biological samples with alcohol and then transferring them into a mounting medium that enhances optical clarity. This improved protocol enables high-resolution imaging of thick biological structures, which was previously challenging. By optimizing this preparation method, we hope to expand the use of SR microscopy for studying large biological samples, helping scientists better understand complex biological structures.

Detection of tea leaf blight in UAV remote sensing images by integrating super-resolution and detection networks.

Tea leaf blight (TLB) is a common disease of tea plants and is widely distributed in tea gardens. Although the use of unmanned aerial vehicle (UAV) remote sensing can help to achieve a wider scale for TLB detection, the blurring of UAV images, overlapping of tea leaves, and small size of TLB spots pose significant challenges to the task of detection. This study proposes a method of detecting TLB in UAV remote sensing images by integrating super-resolution (SR) and detection networks. We use an SR network called SERB-Swin2sr to reconstruct the detailed features of UAV images and solve the problem of detail loss caused by the blurring in UAV images. In SERB-Swin2sr, a squeeze-and-excitation ResNet block (SERB) is introduced to enhance the models' ability to extract the target details in the images, and the convolution stem replaces the convolution block in order to increase the convergence rate and stability of the network. A detection network called SDDA-YOLO is applied to achieve precise detection of TLB in UAV remote sensing images. In SDDA-YOLO, a shuffle dual-dimensional attention (SDDA) module is introduced to enhance the feature fusion capability of the network, and an Xsmall-scale detection layer is used to enhance the detection ability of small lesions. Experimental results show that the proposed method is superior to current detection methods. Compared with a baseline YOLOv8 model, the precision, mAP@0.5, and mAP@0.5:0.95 of the proposed method are improved by 4.2%, 1.6%, and 1.8%, and the size of our model is only 4.6MB.

Research on the fusion imaging method of sign coherence and time reversal for Lamb wave sparse array

Time-reversal imaging struggles to detect plate-like structures due to interference from Lamb wave mode conversion and the processing demands, leading to less effective outcomes. This paper proposes a sign coherence factor and time reversal fusion (SCF-TR) imaging method based on amplitude and phase estimation. This method removes the coherence of array signals during signal reversal and refocusing. It reintroduces the sign coherence component to reduce interference from non-target scattered waves and partially overcome the constraints imposed by the Rayleigh criterion. The method allows imaging at a resolution smaller than the wavelength of Lamb and enhances the quality of the resulting images. In addition, a sparse array design utilizing the White Shark Optimisation Algorithm (WSO) is proposed to streamline the SCF-TR calculation process. This design utilizes sparse full matrix data to improve imaging efficiency. The experimental results show that for single blind hole defects, the SCF-TR method improves the array performance metrics and signal-to-noise ratio by 22.46% and 42.50%, respectively, compared to the TR method. For multiple asymmetric blind hole defects, when the defect size exceeds the resolution threshold, SCF-TR accurately reflects the position and morphology of defects smaller than the wavelength. When the defect size is below the resolution threshold, SCF-TR achieves super-resolution imaging. The sparse array designed using the White Shark Optimization algorithm demonstrates good sidelobe characteristics, effectively reducing sidelobe noise without reducing the array aperture. Moreover, the SCF-TR imaging time is reduced by approximately half while maintaining imaging accuracy.

Extended-depth of field random illumination microscopy, EDF-RIM, provides super-resolved projective imaging

The ultimate aim of fluorescence microscopy is to achieve high-resolution imaging of increasingly larger biological samples. Extended depth of field presents a potential solution to accelerate imaging of large samples when compression of information along the optical axis is not detrimental to the interpretation of images. We have implemented an extended depth of field (EDF) approach in a random illumination microscope (RIM). RIM uses multiple speckled illuminations and variance data processing to double the resolution. It is particularly adapted to the imaging of thick samples as it does not require the knowledge of illumination patterns. We demonstrate highly-resolved projective images of biological tissues and cells. Compared to a sequential scan of the imaged volume with conventional 2D-RIM, EDF-RIM allows an order of magnitude improvement in speed and light dose reduction, with comparable resolution. As the axial information is lost in an EDF modality, we propose a method to retrieve the sample topography for samples that are organized in cell sheets.

NanoPlex: a universal strategy for fluorescence microscopy multiplexing using nanobodies with erasable signals

Fluorescence microscopy has long been a transformative technique in biological sciences. Nevertheless, most implementations are limited to a few targets, which have been revealed using primary antibodies and fluorescently conjugated secondary antibodies. Super-resolution techniques such as Exchange-PAINT and, more recently, SUM-PAINT have increased multiplexing capabilities, but they require specialized equipment, software, and knowledge. To enable multiplexing for any imaging technique in any laboratory, we developed NanoPlex, a streamlined method based on conventional antibodies revealed by engineered secondary nanobodies that allow the selective removal of fluorescence signals. We develop three complementary signal removal strategies: OptoPlex (light-induced), EnzyPlex (enzymatic), and ChemiPlex (chemical). We showcase NanoPlex reaching 21 targets for 3D confocal analyses and 5–8 targets for dSTORM and STED super-resolution imaging. NanoPlex has the potential to revolutionize multi-target fluorescent imaging methods, potentially redefining the multiplexing capabilities of antibody-based assays.

AIDM-CT: software for cone-beam X-ray tomography and deep-learning-based analysis

ABSTRACT Non-destructive testing (NDT) with high-resolution cone-beam X-ray computed tomography (XCT) plays a crucial role in revealing the 3D distribution and morphology of defects within an object. It’s necessary to generalise an intelligent and customised XCT-based NDT protocol, providing solutions for reconstruction quality improvement and complex defect quantitative analysis for users. We analysed the key techniques of XCT and developed the systematic software for data acquisition, reconstruction, intelligent super-resolution and segmentation in our advanced imaging and data mining laboratory (AIDM-CT). The experimental results demonstrated the software efficiently achieves the control of XCT scanning, artefact correction algorithms related to cone-beam geometrics, beam hardening, ring and radial artefacts, deep-learning based slice super-resolution and feature segmentation, and quantitative analysis. The software is programmed by Python language to provide the friendly multi-function graphical user interface (GUI), in which the programming of reconstruction and corrections combined with Computer Unified Device Architecture (CUDA) acceleration to guarantee the high efficiency of XCT task. Particularly, the multi-functional super resolution module (MF-GAN) is proposed to optimise the quality of CT image and the improved U-Net segmentation module (CBAM U-Net) is also developed to fulfill the high-precision quantitative non-destructive testing of homogeneous and variform microdefects in our AIDM-CT software.

Super-resolution imaging reveals the role of DDR1 cluster in NSCLC proliferation

Discoidin domain receptor 1 (DDR1), a transmembrane protein, is crucial in tumor development. Prior studies have demonstrated a significant correlation between protein cluster distribution on the cell membrane and tumor evolution. However, the precise spatial distribution characteristics of DDR1 on cell membranes and their impact on tumor development remain unclear. In this study, we conducted gene expression analysis to investigate DDR1 expression in non-small cell lung cancer (NSCLC) and its association with patient prognosis. We also employed direct stochastic optical reconstruction microscopy (dSTORM) imaging to examine DDR1's spatial distribution in NSCLC cells and tissues. Our findings indicate that DDR1 forms larger, tighter, and more abundant clusters in cancer cells and tissues compared to their normal counterparts. Notably, we observed that the enhanced aggregation of DDR1 clusters increased the likelihood of interaction with SRC, thereby activating the SRC-STAT3 signaling pathway in NSCLC cells and promoting cell proliferation. This study provides novel insights into the role of DDR1 aggregation in tumor proliferation, confirms DDR1 as a potential tumor marker, and serves as a valuable resource for future drug development.

Retraction Note: Optical super-resolution imaging based on improved planning algorithm in sports teaching simulation

Retraction Note: Optical super-resolution imaging based on improved planning algorithm in sports teaching simulation

α1C S1928 Phosphorylation of CaV1.2 Channel Controls Vascular Reactivity and Blood Pressure.

Increased vascular CaV1.2 channel function causes enhanced arterial tone during hypertension. This is mediated by elevations in angiotensin II/protein kinase C signaling. Yet, the mechanisms underlying these changes are unclear. We hypothesize that α1C phosphorylation at serine 1928 (S1928) is a key event mediating increased CaV1.2 channel function and vascular reactivity during angiotensin II signaling and hypertension. The hypothesis was examined in freshly isolated mesenteric arteries and arterial myocytes from control and angiotensin II-infused mice. Specific techniques include superresolution imaging, proximity ligation assay, patch-clamp electrophysiology, Ca2+ imaging, pressure myography, laser speckle imaging, and blood pressure telemetry. Hierarchical "nested" and appropriate parametric or nonparametric t test and ANOVAs were used to assess statistical differences. We found that angiotensin II redistributed the CaV1.2 pore-forming α1C subunit into larger clusters. This was correlated with elevated CaV1.2 channel activity and cooperativity, global intracellular Ca2+ and contraction of arterial myocytes, enhanced myogenic tone, and altered blood flow in wild-type mice. These angiotensin II-induced changes were prevented/ameliorated in cells/arteries from S1928 mutated to alanine knockin mice, which contain a negative modulation of the α1C S1928 phosphorylation site. In angiotensin II-induced hypertension, increased α1C clustering, CaV1.2 activity and cooperativity, myogenic tone, and blood pressure in wild-type cells/tissue/mice were averted/reduced in S1928 mutated to alanine samples. Results suggest an essential role for α1C S1928 phosphorylation in regulating channel distribution, activity and gating modality, and vascular function during angiotensin II signaling and hypertension. Phosphorylation of this single vascular α1C amino acid could be a risk factor for hypertension that may be targeted for therapeutic intervention.

Image Super-Resolution Reconstruction Based on Dense Residual Attention and Multi-Scale Feature Fusion

In order to obtain super-resolution images with richer details and clearer textures, a method for image super-resolution reconstruction using dense residual attention and multi-scale fusion is proposed. First, different scale convolutions are used to fully extract shallow features of the image; then high-frequency features of the image are extracted through one three-layer cascaded multi-scale feature fusion and dense residual attention module, and the reuse of feature map is achieved; finally, residual branches are used to introduce shallow features and high-frequency features of each channel image, and the high-resolution images are reconstructed through up-sampling and sub-pixel convolution. The test results on the Set5, Set14, Bsd100, and Urban100 datasets show that the PSNR and SSIM of our model are superior to most current algorithms, especially in the case of [Formula: see text] reconstruction results. PSNR has improved by 0.2 dB on the Set5 and Bsd100 datasets, and the algorithm has a better subjective visual effect.

OGRN: optical-guided residual dense network for SAR image super-resolution reconstruction network

ABSTRACT Although Convolutional Neural Networks have significantly improved the development of SAR image super-resolution (SR) technology in recent years, it is a very challenging problem to reconstruct SAR image with large-scale factors, such as ×4 and ×8 due to limited available information from low-resolution image. The co-registered high-resolution optical image has been successfully applied to enhance the quality of SAR image due to its discriminative characteristics. Compared with single-frame SAR image SR reconstruction technology, optical image-guided SAR image SR reconstruction technology has better performance. This paper proposes an optical-guided residual dense network (OGRN) for SAR image super-resolution reconstruction network. The network is an end-to-end network. Its most important characteristic is that the SR model of SAR images can be obtained through optical image assistance during training, and high-quality SAR images can be generated without optical image assistance during testing. Firstly, a new dense residual backbone network that extracts high-frequency information is constructed. It can extract and propagate high-frequency features of SAR images efficiently to improve the feature representation ability of the network for high-frequency features. Then, the extracted high-frequency features of the SAR image are used to generate an optical image through the designed SAR-to-Optical image translation network. Finally, a reconstruction module is constructed based on the proposed fusion module for optical and SAR images. The fusion module fuses the high-frequency feature information of the extracted optical images with the SAR image features to provide features for SAR image SR reconstruction. Extensive experiments conducted on Sen1-2 and QXS datasets demonstrated that under the guidance of optical image, our OGRN can achieve excellent performance in both quantitative assessment metrics and visual quality.

Superresolution imaging using superoscillatory diffractive neural networks

Superresolution imaging using superoscillatory diffractive neural networks

Lensless super-resolved quantitative phase imaging with cm-level lateral translations of two glass plates.

In this Letter, we propose a new, to the best of our knowledge, lensless on-chip holographic microscopy platform, which can acquire sub-pixel-shifting holograms through centimeter (cm)-level lateral translations. An LED light source is used to illuminate the sample, and two orthogonally tilted step-structure glass plates are inserted into the optical path. By merely displacing the glass plates under cm-level precision, a series of holograms with sub-pixel displacements can be obtained. Combined with our improved pixel super-resolution (PSR) algorithm, high-quality PSR phase imaging can be achieved. Tests on the high-resolution USAF1951 target demonstrate that the system can achieve a half-width resolution of 870 nm by a camera with a pixel size of 1.67 µm. Additionally, imaging experiments were conducted on phase-type sinusoidal gratings, yeasts, red blood cells, and lilium ovary sections, respectively. The results show that the system can achieve large field-of-view, high-resolution phase imaging under low-cost hardware conditions and holds promise for its applications in biology and medicine.

Super-resolution cropland mapping with Sentinel-2 images based on a self-training learning network

ABSTRACT The Sentinel-2 image is widely used for cropland mapping, but the limitation of its 10 m spatial resolution may significantly impact the accuracy of result in areas characterized by severe cropland fragmentation. To address the issue, this letter proposes a super-resolution cropland mapping model for Sentinel-2 images. The proposed model improves the spatial resolution of Sentinel-2 images to 2.5 m through self-training learning with the Swin Transformer for Image Restoration (SwinIR) model without needing fine-resolution training samples. Then, the random forest classification is applied to map cropland from the super-solved 2.5 m resolution image. The proposed model was assessed in the Jianghan Plain, China, and the results show that the super-resolution images produce cropland maps with higher accuracy than the original Sentinel-2 images. Comparing cropland mapping results before and after super-resolution, the overall accuracy improves from 0.82 to 0.89, while the commission error and omission error decrease from 0.09 to 0.05 and 0.17 to 0.11, respectively. This method addresses the shortage of training samples collection and improves the accuracy of cropland mapping, contributing to more reliable agricultural remote sensing applications.

Architecture-driven quantitative nanoscopy maps cytoskeleton remodeling

Cytoskeleton remodeling which generates force and orchestrates signaling and trafficking to govern cell migration remains poorly understood, partly due to a lack of an investigation tool with high system flexibility, spatiotemporal resolution, and computational sensitivity. Herein, we developed a multimodal superresolution imaging system-based architecture-driven quantitative (ADQ) framework in spatiotemporal-angular hyperspace to enable both identification of the optimal imaging mode with well-balanced fidelity and phototoxicity and accurate postcharacterization of microtubule remodeling. In the ADQ framework, a pixel/voxel-wise metric reflecting heterogeneous intertubule alignment was proposed with improved sensitivity over previous efforts and further incorporated with temporal features to map dynamic microtubule rearrangements. The ADQ framework was verified by assessing microtubule remodeling in drug-induced (de)polymerization, lysosome transport, and migration. Different remodeling patterns from two migration modes were successfully revealed by the ADQ framework, with a front-rear polarization for individual directed migration and a contact site-centered polarization for cell-cell interaction-induced migration in an immune response model. Meanwhile, these migration modes were found to have consistent orientation changes, which exhibited the potential of predicting migration trajectory.

Image super-resolution based on improved ESRGAN and its application in camera calibration

Camera calibration is a key technique in the field of computer vision. The Zhang’s method is currently the most commonly used camera calibration method, but its accuracy is mainly dependent on the image quality, thusly constrained by camera hardware conditions. Considering that high-performance cameras are often expensive, a cost-effective method to improve the quality of calibration images is of great practical value. Without changing the existing hardware conditions, image super-resolution can be considered to enhance the quality of the calibration image. Image super-resolution (SR) is the process of reconstructing an image from low resolution (LR) to high resolution (HR). It can enhance the clarity and detail of the calibration images, which will be beneficial in improving the accuracy of camera calibration. However, there is very little research on the application of image super-resolution to camera calibration. Therefore, this study innovatively proposed a method and process for applying image super-resolution in camera calibration. Firstly, the ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) was optimized and improved, followed by the proposal of the MMRSRGAN (Multi-scale Multi-adaptive-weights Residual Super-Resolution Generative Adversarial Networks). Two methods for local image super-resolution were also proposed for efficient training and testing of datasets, and two evaluation metrics based on reprojection error were proposed evaluate the effectiveness of the proposed model. Training and testing experiments of image super-resolution for camera calibration were conducted on both virtual and real image datasets. The proposed MMRSRGAN was compared with several advanced image super-resolution networks developed in recent years across multiple dimensions and indicators. The positive results have demonstrated the feasibility of applying image super-resolution to camera calibration and showcased the good performance of the MMRSRGAN.

Super-resolution enhancement and segmentation for digital rock based on multi-task joint deep neural network

Reliable segmentation of pores and minerals from high-resolution (HR) digital rock data is the fundamental prerequisite for accurately characterizing the digital rock's physical properties. Limited by the complexity of the super-resolution (SR) issues and the user bias for segmentation, conventional SR enhancement and segmentation methods are generally brutal in satisfying the research demands. In recent years, with the introduction of deep learning technology into digital rock research, deep learning-based approaches for SR enhancement or segmentation methods have emerged and achieved significantly better effects than conventional approaches. Most deep learning-based digital rock SR enhancement and segmentation processing steps are currently separate. Still, preliminary studies have demonstrated that an end-to-end approach that integrates the two steps could achieve better performance. However, the training cost of 3D deep neural networks that can be directly applied to 3D digital rock processing is usually expensive. In contrast, 2D networks cannot be effectively applied to SR segmentation of 3D digital rocks. Here, we present an end-to-end SR segmentation framework for 3D digital rock based on a 2D multi-task joint deep neural network. The multi-task joint networks utilize a parallel architecture that integrates the SegNet for the segmentation task and the EDSR for the SR task. We also improved the loss function to address the issue of category imbalance and proposed an approach to utilize the 2D network for 3D digital rock processing. We demonstrate the effectiveness of the proposed network framework, the improved loss function, and the 3D digital rock processing strategies through ablation experiments on the high and low-resolution (LR) CT image datasets captured by imaging devices. The results show that the evaluation metrics and physical properties of the SR-segmented 2D\\3D digital rocks align with the HR-segmented results (ground truth). It indicates that the proposed framework can improve the performance of digital rock SR enhancement and segmentation and that it is essential to integrate deep learning frameworks into digital rock analysis.

Foreground-background separation and deblurring super-resolution method

The limited depth of field (DOF) inherent in cameras often results in defocused and blurry backgrounds when capturing images in large aperture mode. This not only leads to the loss of crucial background information but also impedes the efficient reconstruction of the background regions. Usually, super-resolution (SR) techniques struggle to produce high-quality results for images captured with large apertures. To enhance the reconstruction quality of defocused regions in large aperture images, a foreground-background separation and deblurring super-resolution (FBSDSR) method was proposed in this paper. Based on the idea of foreground-background separation processing, we first divide the large aperture image into a sharp foreground region (If) and a blurry background region (Ib) based on depth information. The background region (Ib) is then deblurred using an end-to-end iterative filter adaptive network (IFAN). This deblurring process refocuses the background, ultimately restoring an image with sharp details throughout. Finally, the enhanced super-resolution generative adversarial networks (Real-ESRGAN) which specializes in images SR of realistic scenes was used to process the sharp all-in-focus image. This method results in high-quality reconstructions of both the foreground and background of large aperture images. The experimental results demonstrated that the proposed method achieved effective reconstruction of the entire large aperture images clearly, overcoming the limitations of existing methods that struggle to reconstruct defocused regions. This significantly enhances the quality and resolution of large aperture images. Specifically, when FBSDSR is integrated with Real-ESRGAN, the PSNR, LPIPS, NIQE, and hyperIQA metrics were improved by approximately 2.2 %, 45.1 %, 34.7 %, and 10.9 % respectively.