Super-resolution Research Articles

Forest Fire Burn Scar Mapping Based on Modified Image Super-Resolution Reconstruction via Sparse Representation

Forest Fire Burn Scar Mapping Based on Modified Image Super-Resolution Reconstruction via Sparse Representation

Neural Operator for Planetary Remote Sensing Super-Resolution with Spectral Learning

High-resolution planetary remote sensing imagery provides detailed information for geomorphological and topographic analyses. However, acquiring such imagery is constrained by limited deep-space communication bandwidth and challenging imaging environments. Conventional super-resolution methods typically employ separate models for different scales, treating them as independent tasks. This approach limits deployment and real-time applications in planetary remote sensing. Moreover, capturing global context is crucial in planetary remote sensing images due to their contextual similarities. To address these limitations, we propose Discrete Cosine Transform (DCT)–Global Super Resolution Neural Operator (DG-SRNO), a global context-aware arbitrary-scale super-resolution model. DG-SRNO achieves super-resolution at any scale using a single framework by learning the mapping between low-resolution (LR) and high-resolution (HR) function spaces. We mathematically prove the global receptive field of DG-SRNO. To evaluate DG-SRNO’s performance in planetary remote sensing tasks, we introduce the Ceres 800 dataset, a planetary remote sensing super-resolution dataset. Extensive quantitative and qualitative experiments demonstrate DG-SRNO’s impressive reconstruction capabilities.

Impact of photobleaching on quantitative, spatio-temporal, super-resolution imaging of mitochondria in live C. elegans larvae

Super-resolution (SR) 3D rendering allows superior quantitative analysis of intracellular structures but has largely been limited to fixed or ex vivo samples. Here we developed a method to perform SR live imaging of mitochondria during post-embryonic development of C. elegans larvae. Our workflow includes the drug-free mechanical immobilisation of animals using polystyrene nanobeads, which has previously not been used for in vivo SR imaging. Based on the alignment of moving objects and global threshold-based image segmentation, our method enables an efficient 3D reconstruction of individual mitochondria. We demonstrate for the first time that the frequency distribution of fluorescence intensities is not affected by photobleaching, and that global thresholding alone enables the quantitative comparison of mitochondria along timeseries. Our composite approach significantly improves the study of biological structures and processes in SR during C. elegans post-embryonic development. Furthermore, the discovery that image segmentation does not require any prior correction against photobleaching, a fundamental problem in fluorescence microscopy, will impact experimental strategies aimed at quantitatively studying the dynamics of organelles and other intracellular compartments in any biological system.

Stochastically structured illumination microscopy scan less super resolution imaging

In super-resolution, a varying illumination image stack is required. This enriched dataset typically necessitates precise mechanical control and micron-scale optical alignment and repeatability. Here, we introduce a novel methodology for super-resolution microscopy called stochastically structured illumination microscopy (S2IM), which bypasses the need for illumination control exploiting instead the random, uncontrolled movement of the target object. We tested our methodology within the clinically relevant ophthalmoscopic setting, harnessing the inherent saccadic motion of the eye to induce stochastic displacement of the illumination pattern on the retina. We opted to avoid human subjects by utilizing a phantom eye model featuring a retina composed of human induced pluripotent stem cells (iPSC) retinal neurons and replicating the ocular saccadic movements by custom actuators. Our findings demonstrate that S2IM unlocks scan-less super-resolution with a resolution enhancement of 1.91, with promising prospects also beyond ophthalmoscopy applications such as active matter or atmospheric/astronomical observation.

The transmembrane domain of the desmosomal cadherin desmoglein-1 governs lipid raft association to promote desmosome adhesive strength.

Cholesterol- and sphingolipid-enriched domains called lipid rafts are hypothesized to selectively coordinate protein complex assembly within the plasma membrane to regulate cellular functions. Desmosomes are mechanically resilient adhesive junctions that associate with lipid raft membrane domains, yet the mechanisms directing raft association of the desmosomal proteins, particularly the transmembrane desmosomal cadherins, are poorly understood. We identified the desmoglein-1 (DSG1) transmembrane domain (TMD) as a key determinant of desmoglein lipid raft association and designed a panel of DSG1TMD variants to assess the contribution of TMD physicochemical properties (length, bulkiness, and palmitoylation) to DSG1 lipid raft association. Sucrose gradient fractionations revealed that TMD length and bulkiness, but not palmitoylation, govern DSG1 lipid raft association. Further, DSG1 raft association determines plakoglobin recruitment to raft domains. Super-resolution imaging and functional assays uncovered a strong relationship between the efficiency of DSG1TMD lipid raft association and the formation of morphologically and functionally robust desmosomes. Lipid raft association regulated both desmosome assembly dynamics and DSG1 cell surface stability, indicating that DSG1 lipid raft association is required for both desmosome formation and maintenance. These studies identify the biophysical properties of desmoglein transmembrane domains as key determinants of lipid raft association and desmosome adhesive function.

Clickable HaloTag ligands for live cell labeling and imaging

Self-labeling protein (SLP) tags, such as HaloTag, have gained considerable interest as advanced tools for live cell labeling. However, the chloroalkane-based substrates that can be directly used for protein labeling are limited. Here, we report two bioorthogonal small molecule linkers, chloroalkane-tetrazine (CA-Tz) and chloroalkane-azide (CA-N3), which can penetrate cell membranes and facilitate click chemistry-based labeling in live cells. We compare their labeling capability using two clickable silicon rhodamine dyes (SiR-PEG3-TCO and SiR-PEG4-DBCO). Confocal imaging results demonstrate that using CA-Tz and SiR-PEG3-TCO dye exhibits superior intracellular labeling with low nonspecific signals. We subsequently compared the photostability of SiR dyes with that of green fluorescent proteins (mEmerald). Total internal reflection fluorescence (TIRF) imaging indicates that SiR dyes exhibit superior photostability under identical excitation conditions, making them suitable for long-term cell imaging. Furthermore, SiR dyes labeling also shows high structure retention for the fourth-order super-resolution optical fluctuation imaging (SOFI) compared to fluorescent proteins. This study presents clickable HaloTag linkers as effective tools for live cell labeling and imaging, highlighting the high-quality labeling of chloroalkane linkers and clickable dyes for live cell imaging.

Airborne Multi-Channel Forward-Looking Radar Super-Resolution Imaging Using Improved Fast Iterative Interpolated Beamforming Algorithm

Radar forward-looking imaging is critical in many civil and military fields, such as aircraft landing, autonomous driving, and geological exploration. Although the super-resolution forward-looking imaging algorithm based on spectral estimation has the potential to discriminate multiple targets within the same beam, the estimation of the angle and magnitude of the targets are not accurate due to the influence of sidelobe leakage. This paper proposes a multi-channel super-resolution forward-looking imaging algorithm based on the improved Fast Iterative Interpolated Beamforming (FIIB) algorithm to solve the problem. First, the number of targets and the coarse estimates of angle and magnitude are obtained from the iterative adaptive approach (IAA). Then, the accurate estimates of angle and magnitude are achieved by the strategy of iterative interpolation and leakage subtraction in FIIB. Finally, a high-resolution forward-looking image is obtained through non-coherent accumulation. The simulation results of point targets and scenes show that the proposed algorithm can distinguish multiple targets in the same beam, effectively improve the azimuthal resolution of forward-looking imaging, and attain the accurate reconstruction of point targets and the contour reconstruction of extended targets.

Long-Axial-Range Double-Helix Point Spread Functions for 3D Volumetric Super-Resolution Imaging.

Single-molecule localization microscopy (SMLM) is a powerful tool for observing structures beyond the diffraction limit of light. Combining SMLM with engineered point spread functions (PSFs) enables 3D imaging over an extended axial range, as has been demonstrated for super-resolution imaging of various cellular structures. However, super-resolving structures in 3D in thick samples, such as whole mammalian cells, remains challenging as it typically requires acquisition and postprocessing stitching of multiple slices to cover the entire sample volume or more complex analysis of the data. Here, we demonstrate how the imaging and analysis workflows can be simplified by 3D single-molecule super-resolution imaging with long-axial-range double-helix (DH)-PSFs. First, we experimentally benchmark the localization precisions of short- and long-axial-range DH-PSFs at different signal-to-background ratios by imaging fluorescent beads. The performance of the DH-PSFs in terms of achievable resolution and imaging speed was then quantified for 3D single-molecule super-resolution imaging of mammalian cells by DNA-PAINT imaging of nuclear lamina protein lamin B1 in U-2 OS cells. Furthermore, we demonstrate how the use of a deep-learning-based algorithm allows the localization of dense emitters, drastically improving the achievable imaging speed and resolution. Our data demonstrate that using long-axial-range DH-PSFs offers stitching-free, 3D super-resolution imaging of whole mammalian cells, simplifying the experimental and analysis procedures for obtaining volumetric nanoscale structural information.

GAN-Based Dual Image Super Resolution for Satellite Imagery Decreasing Radiometric Uncertainty

Abstract. Super resolution for satellite imagery possesses specific challenges due to its unique feature geometry compared to classic computer vision. Specifically, satellite images contain a high amount of relatively small, distributed high-frequency features that are hard to preserve when sampling up to a higher resolution. General adversarial networks (GANs) are suitable for super resolution tasks but need special attention concerning the geometric and radiometric accuracy of the results. We propose a network for versatile satellite imagery super resolution (VSISR) that focuses on high-frequency detail preservation and radiometric consistency. As a novelty, it is able to utilize a reference high resolution image during inference to lower hallucinations without altering the source image’s radiometric characteristics.A GAN that is already handling well high frequencies in standard computer vision cases is adapted for satellite imagery and supplemented with a mixed pixel approach for data augmentation. Training on a diverse RGB dataset from four satellite missions results in a versatile super resolution model that is optimized to preserve radiometric features and minimize hallucinations. Compared to other neural networks for satellite image super resolution in their respective datasets, VSISR performs in the middle regarding mean PSNR (25.30 dB) and SSIM (0.8098). Nevertheless, it is the first time that, using mixed pixel training on a comparably small dataset, this versatility concerning the satellite data source is achieved while maintaining their unique radiometric traits.

Submicrometre spatiotemporal characterization of the Toxoplasma adhesion strategy for gliding motility.

Toxoplasma gondii is a protozoan apicomplexan parasite that uses an adhesion-dependent mode of motility termed gliding to access host cells and disseminate into tissues. Previous studies on Apicomplexa motile morphotypes, including the T. gondii tachyzoite, have identified a cortical actin-myosin motor system that drives the rearward translocation of transmembrane adhesins, thus powering forward movement. However, this model is currently questioned. Here, combining micropatterning and tunable surface chemistry (to edit parasite surface ligands) with flow force and live or super-resolution imaging, we show that tachyzoites build only one apical anchoring contact with the substrate, over which it slides. Furthermore, we show that glycosaminoglycan-parasite interactions are sufficient to promote such force-productive contact and find that the apicobasal flow is set up independent of adhesin release and surface interactions. These findings should enable further characterization of the molecular functions at the T. gondii-substrate mechanosensitive interface and their comparison across apicomplexans.

Piezo2 interacts with E-cadherin in specialized gastrointestinal epithelial mechanoreceptors.

Piezo2 is a mechanically gated ion channel most commonly expressed by specialized mechanoreceptors, such as the enteroendocrine cells (EECs) of the gastrointestinal epithelium. A subpopulation of EECs expresses Piezo2 and functionally resembles the skin's touch sensors, called Merkel cells. Low-magnitude mechanical stimuli delivered to the mucosal layer are primarily sensed by mechanosensitive EECs in a process we term "gut touch." Piezo2 transduces cellular forces into ionic currents, a process that is sensitive to bilayer tension and cytoskeletal depolymerization. E-cadherin is a widely expressed protein that mediates cell-cell adhesion in epithelia and interacts with scaffold proteins that anchor it to actin fibers. E-cadherin was shown to interact with Piezo2 in immortalized cell models. We hypothesized that the Piezo2-E-cadherin interaction is important for EEC mechanosensitivity. To test this, we used super-resolution imaging, co-immunoprecipitation, and functional assays in primary tissues from mice and gut organoids. In tissue EECs and intestinal organoids, we observed multiple Piezo2 cellular pools, including one that overlaps with actin and E-cadherin at the cells' lateral walls. Further, E-cadherin co-immunoprecipitated with Piezo2 in the primary colonic epithelium. We found that E-cadherin knockdown decreases mechanosensitive calcium responses in mechanically stimulated primary EECs. In all, our results demonstrate that Piezo2 localizes to the lateral wall of EECs, where it physically interacts with E-cadherin and actin. They suggest that the Piezo2-E-cadherin-actin interaction is important for mechanosensitivity in the gut epithelium and possibly in tissues where E-cadherin and Piezo2 are co-expressed in epithelial mechanoreceptors, such as skin, lung, and bladder.

HVASR: Enhancing 360-degree video delivery with viewport-aware super resolution

In recent years, 360-degree videos have gained significant traction due to their capacity to provide immersive experiences. However, the adoption of 360-degree videos substantially escalates bandwidth demands, necessitating approximately four to ten times more bandwidth than traditional video formats do. This presents a considerable challenge in maintaining high-quality videos in environments characterized by limited bandwidth or unstable networks. A trend has emerged where client-side computational power and deep neural networks are employed to enhance video quality while mitigating bandwidth requirements within contemporary video delivery systems. These approaches segment a video into discrete chunks and apply super resolution (SR) models to each segment, streaming low-resolution (LR) chunks alongside their corresponding SR models to the client. Although these methods enhance both video quality and transmission efficiency for conventional videos, they impose greater computational resource demands when applied to 360-degree content, thereby constraining widespread implementation. This paper introduces an innovative method called HVASR for 360-degree videos that leverages viewport information for more precise segmentation and minimizes model training costs as well as bandwidth requirements. Additionally, HVASR incorporates a viewport-aware training strategy that is aimed at further enhancing performance while reducing computational expenses. The experimental results demonstrate that HVASR achieves an average utility increase ranging from 12.46% to 40.89% across various scenes.

Learning face super-resolution through identity features and distilling facial prior knowledge

Deep learning techniques in electronic surveillance have shown impressive performance for super-resolution (SR) of captured low-quality face images. Most of these techniques adopt facial priors to improve the feature details in the resultant super-resolved images. However, the estimation of facial priors from the captured low-quality images is often inaccurate in real-life situations because of their tiny, noisy, and blurry nature. Thus, the fusion of such priors badly affects the performance of these models. Therefore, this work presents a teacher–student-based face SR framework that efficiently preserves the personal facial structure information in the super-resolved faces. In the proposed framework, the teacher network exploits the facial heatmap-based ground-truth-prior to learn the facial structure that is utilized by the student network. The student network is trained with the identity feature loss for maintaining the identity and facial structure information in reconstructed high-resolution (HR) face images. The performance of the proposed framework is evaluated by conducting the experimental study on standard datasets namely CelebA-HQ and LFW face. The experimental results reveal that the proposed technique conquers the existing methods for the face SR task.

Learning super-resolution and pyramidal convolution residual network for vehicle re-identification

Vehicle re-identification (Vehicle Re-ID) aims at retrieving and tracking the specified target vehicle with multiple other cameras, which can provide help in checking violations and catching fugitives, but there are still the following problems that need to be solved urgently. First, the existing collected Vehicle Re-ID data often have low resolution and blur in local regions, so that the Vehicle Re-ID algorithm cannot accurately extract subtle feature representations. In addition, small features are easy to cause the disappearance of features under the operation of a large convolution kernel, which makes the model unable to capture and learn subtle features, resulting in inaccurate judgment of vehicles. In this study, we propose a Vehicle Re-ID method based on super resolution and pyramidal convolution residual network. Firstly, a super-resolution image generation network leveraging generative adversarial networks (GANs) is proposed. This network employs both content loss and adversarial loss as optimization criteria, ensuring an efficient transformation from a low-resolution image into a super-resolution counterpart, while meticulously preserving intricate high-frequency details. Then, multi levels of pyramidal convolution operations are designed to generate multi-scale features, which can capture information on different scales. Moreover, the concept of residual learning is applied between the multi levels of pyramidal convolution operations to expedite model optimization and enhance recognition capabilities. Ultimately, the double pyramidal convolutions are meticulously employed on both the original image and the super-resolution image, yielding low-noise feature representations and intricate semantic information respectively. By seamlessly fusing these two diverse sources of information, the resultant combined features exhibit heightened discrimination capabilities and significantly bolster the robustness of image features. In order to verify the effectiveness of the proposed method, extensive experiments are carried out on VeRi-776 and VehicleID datasets. The experimental results show that the method proposed in this paper effectively captures the detail information of vehicle images, accurately distinguishes the subtle differences between different vehicles of the same type, and is superior to state-of-the-art methods.

Super resolution reconstruction of fluorescence microscopy images by a convolutional network with physical priors

Super-solution fluorescence microscopy, such as single-molecule localization microscopy (SMLM), is effective in observing subcellular structures and achieving excellent enhancement in spatial resolution in contrast to traditional fluorescence microscopy. Recently, deep learning has demonstrated excellent performance in SMLM in solving the trade-offs between spatiotemporal resolution, phototoxicity, and signal intensity. However, most of these researches rely on sufficient and high-quality datasets. Here, we propose a physical priors-based convolutional super-resolution network (PCSR), which incorporates a physical-based loss term and an initial optimization process based on the Wiener filter to create excellent super-resolution images directly using low-resolution images. The experimental results demonstrate that PCSR enables the achievement of a fast reconstruction time of 100 ms and a high spatial resolution of 10 nm by training on a limited dataset, allowing subcellular research with high spatiotemporal resolution, low cell phototoxic illumination, and high accessibility. In addition, the generalizability of PCSR to different live cell structures makes it a practical instrument for diverse cell research.

Disentangled feature fusion network for lightweight image super-resolution

Recently, the quality of generated images in image super-resolution (SR) has significantly improved due to the widespread application of convolutional neural networks. Existing super-resolution methods often overlook the importance of network width and instead prioritize designing deeper network structures to improve performance. Deeper networks, however, demand greater computational resources and are more challenging to train, making them less suitable for devices with limited performance. To address this issue, we propose a novel method called the disentangled feature fusion network (DFFN). Specifically, we construct dual feature extraction streams (DFES) using the strategies of feature disentanglement and parameter sharing, which increase the network's width without increasing the number of parameters. To facilitate the interaction of information between the dual feature extraction streams, we design a feature interaction module (FIM) that separately enhances high and low-frequency features. Furthermore, a feature fusion module (FFM) is presented to efficiently fuse different levels of high and low-frequency feature information. The proposed DFFN integrates DFES, FIM, and FFM, which not only widens the network without increasing parameters but also minimizes the loss of crucial feature information. Extensive experiments on five benchmark datasets demonstrate that our method significantly outperforms other state-of-the-art lightweight super-resolution methods, achieving a superior balance between parameter quantity, reconstruction performance, and model complexity. The code is available at https://github.com/zhoujy-aust/DFFN.

Deep Variational Network Toward Blind Image Restoration.

Blind image restoration (IR) is a common yet challenging problem in computer vision. Classical model-based methods and recent deep learning (DL)-based methods represent two different methodologies for this problem, each with their own merits and drawbacks. In this paper, we propose a novel blind image restoration method, aiming to integrate both the advantages of them. Specifically, we construct a general Bayesian generative model for the blind IR, which explicitly depicts the degradation process. In this proposed model, a pixel-wise non-i.i.d. Gaussian distribution is employed to fit the image noise. It is with more flexibility than the simple i.i.d. Gaussian or Laplacian distributions as adopted in most of conventional methods, so as to handle more complicated noise types contained in the image degradation. To solve the model, we design a variational inference algorithm where all the expected posteriori distributions are parameterized as deep neural networks to increase their model capability. Notably, such an inference algorithm induces a unified framework to jointly deal with the tasks of degradation estimation and image restoration. Further, the degradation information estimated in the former task is utilized to guide the latter IR process. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.

Super-resolution imaging of cellular pseudopodia dynamics with a target-specific blinkogenic probe

Super-resolution imaging of cellular pseudopodia dynamics with a target-specific blinkogenic probe

Nanoscale Janus particle fabrication and direct observation by super-resolution microscopy

We have developed a new and facile approach for synthesis, labeling, and characterizing amphiphilic nanoscale Janus particles. The aqueous dispersion of the Aerodisp W1226 fumed silica particles were embedded into polycarbonate (P.C.) microspheres surfaces via inverse solvent displacement method and further primary functionalization by an amine group. Subsequently, the polymer was dissolved, and a second modification was preferred to introduce a thiol group on the revealed embedded side of the particles, forming silica-based Janus/amphiphilic particles. We first characterized the Janus particles by laser scanning confocal microscopy and then with direct stochastic optical reconstruction microscopy (dSTORM) to detect the labeling distribution of the different functional groups. Super-resolution imaging results confirmed that the fabricated particles are Janus particles with different chemical properties on each hemisphere. The particles were also characterized by scanning and transmission electron microscopy for size and shape. The results showed aspherical particles with two hemispheres, confirming successful fabrication and chemical modification. Also, this work successfully demonstrated the synthesis, characterization and analysis of nanoscale Janus/amphiphilic particles nanoscale Janus particle.

Multi-modal Ultrafast Sonography Microscopy (MUSM) for super-resolution imaging of cerebral vascular dynamics in a mouse model of hypertension induced by Angiotensin-II and L-NAME

Multi-modal Ultrafast Sonography Microscopy (MUSM) for super-resolution imaging of cerebral vascular dynamics in a mouse model of hypertension induced by Angiotensin-II and L-NAME