Structured Illumination Microscopy Research Articles

A Structured Illumination Microscope with the Combination of Optical Sectioning and Lateral Resolution Enhancement for Precise 3D Industrial Metrology

A Structured Illumination Microscope with the Combination of Optical Sectioning and Lateral Resolution Enhancement for Precise 3D Industrial Metrology

High-speed in vivo calcium recording using structured illumination with self-supervised denoising

High-speed widefield fluorescence imaging of neural activity in vivo is fundamentally limited by fluctuations in recorded signal due to background contamination and stochastic noise. In this study, we show background and shot noise-reduced imaging of the ultrafast genetically encoded Ca2+ indicator GCaMP8f in CA1 pyramidal neurons using periodic structured illumination (SI) with computational image reconstruction. We implement what we believe to be a novel reconstruction method for data acquired using periodic structured illumination, termed pseudo-HiLo (pHiLo), that combines a pseudo-widefield (pWF) reconstruction with individual SI frames to perform a HiLo reconstruction. We compare this new technique to interleaved optical sectioning structured illumination microscopy (OS-SIM) and pWF reconstruction. We quantify the performance of each reconstruction by evaluating contrast, transient peak-to-noise ratio (PNR), pairwise correlation coefficients between ΔF/F time courses extracted from individual in-focus cells, and correlation coefficients between each cell with surrounding cell-free background pixels. We additionally incorporate a self-supervised deep learning method for real-time noise suppression (DeepCAD-RT) into our data preprocessing pipeline. At 500 Hz frame rates, we demonstrate a 75% increase in PNR using the denoised pHiLo reconstruction compared to pWF. Utilizing DeepCAD-RT, we show significant PNR improvements using both structured illumination (SI) reconstruction methods with OS-SIM showing a 59% increase in PNR after denoising. Both pHiLo and OS-SIM reconstructions result in a ≈65% decrease in the mean correlation coefficient of the ΔF/F time courses between ROIs in comparison with pWF, indicating the potential to remove background fluorescent transients from out-of-focus cells.

Clog P-Guided Development of Multi-Colored Buffering Fluorescent Probes for Super-Resolution Imaging of Lipid Droplet Dynamics.

Super-resolution fluorescence imaging of live cells increasingly demands fluorescent probes capable of multi-color and long-term dynamic imaging. Understanding the mechanisms of probe-target recognition is essential for the engineered development of such probes. In this study, it is discovered that the molecular lipid solubility parameter, Clog P, determines the staining performance of fluorescent dyes on lipid droplets (LDs). Fluorescent dyes with Clog P values between 2.5 and 4 can form buffering pools outside LDs, replacing photobleached dyes within LDs to maintain constant fluorescence intensity in LDs, thereby enabling dynamic super-resolution imaging of LDs. Guided by Clog P, four different colored buffering LD probes spanning the visible light spectrum have been developed. Using Structured Illumination Microscopy (SIM), the role of LD dynamics have been tracked during cellular ferroptosis with the secretion, storage, and degradation of overexpressed ACSL3 proteins. It is found that LDs serve as storage sites for these proteins through membrane fusion, and further degrade overexpressed proteins via interactions with organelles like lysosomes or through lipophagy, thereby maintaining cellular homeostasis.

Quantitative evaluation of intensity fidelity of super-resolution reconstruction for structured illumination microscopy

Super-resolution structured illumination microscopy (SR-SIM) relies heavily on post-processing reconstruction to obtain high-quality SR images from raw data. Although many SIM reconstruction algorithms have been developed to recover fine cellular structures with high fidelity even from the noisy data, whether the pixel intensities of reconstructed SR images are still proportional to the original fluorescence intensity has been less explored. The linearity between the intensity before and after reconstruction is defined as the intensity fidelity. Here, we proposed a method to evaluate the reconstructed SR image intensity fidelity at different spatial frequencies. With the proposed metric, we systematically investigated the impact of the key factors on the intensity fidelity in the standard Wiener-SIM reconstructions with simulated data, then evaluated the intensity fidelity of the SR images reconstructed by representative open-source packages. Our work provides a reference for SR-SIM image intensity fidelity improvement.

Modern Methods of Fluorescence Nanoscopy in Biology

Optical microscopy has undergone significant changes in recent decades due to the breaking of the diffraction limit of optical resolution and the development of high-resolution imaging techniques, which are collectively known as fluorescence nanoscopy. These techniques allow researchers to observe biological structures and processes at a nanoscale level of detail, revealing previously hidden features and aiding in answering fundamental biological questions. Among the advanced methods of fluorescent nanoscopy are: STED (Stimulated Emission Depletion Microscopy), STORM (STochastic Optical Reconstruction Microscopy), PALM (Photo-activated Localization Microscopy), TIRF (Total Internal Reflection Fluorescence), SIM (Structured Illumination Microscopy), MINFLUX (Minimal Photon Fluxes), PAINT (Points Accumulation for Imaging in Nanoscale Topography) и RESOLFT (REversible Saturable Optical Fluorescence Transitions) and others. In addition, most of these methods make it possible to obtain volumetric (3D) images of the objects under study. In this review, we will look at the principles of these methods, their advantages and disadvantages, and their application in biological researches.

Addressable scanning multifocal structured illumination microscopy using acousto-optic deflectors.

Multifocal structured illumination microscopy (MSIM) is a popular super-resolution imaging technique known for its good probe compatibility, low laser power requirements, and improved imaging depth, making it widely applicable in biomedical research. However, the speed of MSIM imaging is typically constrained by the approaches employed to generate and scan the laser foci across the sample. In this study, we propose a flexible two-photon excitation MSIM method using a pair of acousto-optic deflectors. By adopting addressable scanning (AS) and synchronized capturing, MSIM super-resolution imaging can be performed in multiple discrete regions of interest (ROIs) within the field of view. Notably, this AS-MSIM scheme not only enhances the speed of MSIM imaging but also alleviates photobleaching and phototoxicity to biological samples. We demonstrate its potential by achieving super-resolution imaging of selected mitochondria within cells at a frame rate of 4 Hz. Furthermore, we deliberate the possibility of even faster imaging.

Tumor microenvironment-based smart beacon drug delivery system for in situ visualization of tumor therapy

Real-time monitoring of the effectiveness of tumor-targeted drugs is crucial for assessing their impact. To address this need, we developed a beacon drug delivery system, MDB, which incorporates tumor targeting, anti-tumor effects, and near-infrared fluorescence emission for on-site visualization of tumor therapy. In non-cancerous cells, MDB remains inactive as a prodrug but is selectively activated within tumor cells when the borate ester group reacts with H2O2 in the tumor microenvironment (TME), releasing the beacon drug MDBRe. MDBRe can down-regulate the expression of the protein heme oxygenase 1 (HO-1), ultimately disrupting the mitochondrial energy supply system and leading to tumor cell death. It also emits near-infrared fluorescence for drug tracking under structured illumination microscopy (SIM). Both in vitro and in vivo experiments confirm the effective delivery of MDBRe by MDB to H2O2-enriched tumor tissues, resulting in a potent anti-tumor effect, visualization of drug distribution in tumors, and monitoring of therapy progress. In conclusion, our approach presents a novel method for targeted drug delivery to tumors and enables real-time visual monitoring of the treatment process.

Large-field optical sectioning structured illumination microscopy

Nowadays, fringe generation and fast-switching in structured illumination microscopy (SIM) is often performed using spatial light modulators (SLMs) and digital micromirror devices (DMDs). Yet, the field of view (FOV) or the spatial bandwidth product (SBP) is constrained by the limited pixel number of the SLMs or DMDs. In this study, we present a large-field optical-sectioning structured illumination microscopy (LF-OS-SIM) scheme, which features a large FOV and fast phase shifting. LF-OS-SIM utilizes a one-dimensional grating to generate stripe patterns and a spatial light modulator (SLM) to perform phase-shifting in the Fourier domain. Therefore, this technique can improve the spatial bandwidth product (SBP) by a factor of 2.6 compared to the conventional OS-SIM under the same experimental parameters. The superiority of LF-OS-SIM is demonstrated by imaging euro coins and fluorescent samples, and the results imply that this method has the potential to be applied for 3D imaging of industrial micro-devices and biosamples.

Physical prior-guided deep learning for SIM reconstruction: modeling object-to-image degradation

Structured illumination microscopy (SIM) provides an enhanced spatial resolution of up to twice the conventional capacity. Recently, many approaches have attempted to combine deep learning frameworks with SIM reconstruction for better capability. Nonetheless, the inadequacy of training samples highlights the challenge of limited credibility and low generalization ability of deep learning, thus significantly constraining the application in biology. To tackle this issue, we propose an object-to-image plane degradation network (OIDN) guided by the physical process of optical imaging. Specifically, the proposed OIDN embeds the object-to-image plane degradation process into the reconstruction network to provide explicit guidance. With a set of learnable point spread function (PSF) parameters constrained by physical prior, OIDN successfully converts the conventional image-to-image data pattern mapping into the object-to-image plane degradation mapping that highly aligns with the optical processes of SIM imaging. Comprehensive experiments demonstrate that the proposed method reliably yields high-quality images across signal-to-noise ratio conditions, exhibiting superior generalization ability across diverse datasets and sample types.

Graphene Microelectrode Arrays, 4D Structured Illumination Microscopy, and a Machine Learning Spike Sorting Algorithm Permit the Analysis of Ultrastructural Neuronal Changes During Neuronal Signaling in a Model of Niemann-Pick Disease Type C.

Simultaneously recording network activity and ultrastructural changes of the synapse is essential for advancing understanding of the basis of neuronal functions. However, the rapid millisecond-scale fluctuations in neuronal activity and the subtle sub-diffraction resolution changes of synaptic morphology pose significant challenges to this endeavor. Here, specially designed graphene microelectrode arrays (G-MEAs) are used, which are compatible with high spatial resolution imaging across various scales as well as permit high temporal resolution electrophysiological recordings to address these challenges. Furthermore, alongside G-MEAs, an easy-to-implement machine learning algorithm is developed to efficiently process the large datasets collected from MEA recordings. It is demonstrated that the combined use of G-MEAs, machine learning (ML) spike analysis, and 4D structured illumination microscopy (SIM) enables monitoring the impact of disease progression on hippocampal neurons which are treated with an intracellular cholesterol transport inhibitor mimicking Niemann-Pick disease type C (NPC), and show that synaptic boutons, compared to untreated controls, significantly increase in size, leading to a loss in neuronal signaling capacity.

Neural space-time model for dynamic multi-shot imaging.

Computational imaging reconstructions from multiple measurements that are captured sequentially often suffer from motion artifacts if the scene is dynamic. We propose a neural space-time model (NSTM) that jointly estimates the scene and its motion dynamics, without data priors or pre-training. Hence, we can both remove motion artifacts and resolve sample dynamics from the same set of raw measurements used for the conventional reconstruction. We demonstrate NSTM in three computational imaging systems: differential phase-contrast microscopy, three-dimensional structured illumination microscopy and rolling-shutter DiffuserCam. We show that NSTM can recover subcellular motion dynamics and thus reduce the misinterpretation of living systems caused by motion artifacts.

Image restoration in frequency space using complex-valued CNNs.

Real-valued convolutional neural networks (RV-CNNs) in the spatial domain have outperformed classical approaches in many image restoration tasks such as image denoising and super-resolution. Fourier analysis of the results produced by these spatial domain models reveals the limitations of these models in properly processing the full frequency spectrum. This lack of complete spectral information can result in missing textural and structural elements. To address this limitation, we explore the potential of complex-valued convolutional neural networks (CV-CNNs) for image restoration tasks. CV-CNNs have shown remarkable performance in tasks such as image classification and segmentation. However, CV-CNNs for image restoration problems in the frequency domain have not been fully investigated to address the aforementioned issues. Here, we propose several novel CV-CNN-based models equipped with complex-valued attention gates for image denoising and super-resolution in the frequency domains. We also show that our CV-CNN-based models outperform their real-valued counterparts for denoising super-resolution structured illumination microscopy (SR-SIM) and conventional image datasets. Furthermore, the experimental results show that our proposed CV-CNN-based models preserve the frequency spectrum better than their real-valued counterparts in the denoising task. Based on these findings, we conclude that CV-CNN-based methods provide a plausible and beneficial deep learning approach for image restoration in the frequency domain.

Quantitative and comparative assessment of dyes and protocols for rapid ex vivo microscopy of fresh tissues

Using ex vivo microscopy, virtual pathology can improve histological procedures by providing pathology images in near real-time without tissue destruction. Several emerging and promising approaches leverage fast-acting small-molecule fluorescent stains to replicate traditional pathology structural contrast, combined with rapid optical sectioning microscopes. However, several vital challenges must be addressed to translate virtual pathology into the clinical environment. One such challenge is selecting robust, reliable, and repeatable staining protocols that can be adopted across institutions. In this work, we addressed the effects of dye selection and staining protocol on image quality in rapid point-of-care imaging settings. For this purpose, we used structured illumination microscopy to evaluate fluorescent dyes currently used in the field of ex vivo virtual pathology, in particular, studying the effects of staining protocol and temporal and photostability on image quality. We observed that DRAQ5 and SYBR gold provide higher image quality than TO-PRO3 and RedDot1 in the nuclear channel and Eosin Y515 in the extracellular/cytoplasmic channel than Atto488. Further, we found that TO-PRO3 and Eosin Y515 are less photostable than other dyes. Finally, we identify the optimal staining protocol for each dye and demonstrate pan-species generalizability.

Enhanced denoising for weak signal preservation in structured illumination microscopy

Structured illumination microscopy (SIM) is a powerful super-resolution technology in biological science because of its fast imaging speed, low phototoxicity, and full-field imaging. Despite this, SIM is hampered by out-of-focus background noise, which can obscure weak fluorescence signals and render them unrecognizable. Previous denoising algorithms tended to eliminate the noise along with the weak signals, causing a decrease in image quality. To address this issue, we propose a denoising algorithm based on out-of-focus plane information extraction (OPIE-SIM) that salvages the weak signal from the out-of-focus background noise. The OPIE-SIM algorithm enhances weak fluorescence signals by combining out-of-focus layer information with focal plane data and correcting the differences in point spread functions (PSF). This approach eliminates out-of-focus background noise and preserves the integrity of weak fluorescence structures while significantly reducing image acquisition time compared to traditional over-focusing imaging techniques. Through extensive simulations and experiments, we verified the feasibility of our approach. Compared with other denoising algorithms, our method generates images with a higher signal-to-noise ratio while maintaining the integrity of weak fluorescence structures.

Genome-wide analysis of the biophysical properties of chromatin and nuclear proteins in living cells with Hi-D.

To understand the dynamic nature of the genome, the localization and rearrangement of DNA and DNA-binding proteins must be analyzed across the entire nucleus of single living cells. Recently, we developed a computational light microscopy technique, called high-resolution diffusion (Hi-D) mapping, which can accurately detect, classify and map diffusion dynamics and biophysical parameters such as the diffusion constant, the anomalous exponent, drift velocity and model physical diffusion from the data at a high spatial resolution across the genome in living cells. Hi-D combines dense optical flow to detect and track local chromatin and nuclear protein motion genome-wide and Bayesian inference to characterize this local movement at nanoscale resolution. Here we present the Python implementation of Hi-D, with an option for parallelizing the calculations to run on multicore central processing units (CPUs). The functionality of Hi-D is presented to the users via user-friendly documented Python notebooks. Hi-D reduces the analysis time to less than 1 h using a multicore CPU with a single compute node. We also present different applications of Hi-D for live-imaging of DNA, histone H2B and RNA polymerase II sequences acquired with spinning disk confocal and super-resolution structured illumination microscopy.

Expanding super-resolution imaging versatility in organisms with multi-confocal image scanning microscopy

Abstract Resolving complex three-dimensional subcellular dynamics noninvasively in live tissues demands imaging tools that balance spatiotemporal resolution, field-of-view and phototoxicity. Image scanning microscopy (ISM), as an advancement of confocal laser scanning microscopy, provides a two-fold 3D resolution enhancement. Nevertheless, the relatively low imaging speed has been the major obstacle for ISM to be further employed in in vivo imaging of biological tissues. Our proposed solution, multi-confocal image scanning microscopy (MC-ISM), aims to overcome the limitations of existing techniques in terms of spatiotemporal resolution balancing by optimizing pinhole diameter and pitch, eliminating out-of-focus signals, and introducing a frame reduction reconstruction algorithm. The imaging speed is increased by 16 times compared with multifocal structured illumination microscopy. We further propose single-galvo scan, akin to the Archimedes spiral in spinning disk confocal system, to ensure high speed ang high accuracy scan without galvanometer's inertial motion. Benefitting from its high photon efficiency, MC-ISM allows continuous imaging of mitochondria dynamics in live cell for 1000 frames without apparent phototoxicity, reaching an imaging depth of 175 μm. Noteworthy, MC-ISM enables the observation of the inner membrane structure of living mitochondria in Arabidopsis hypocotyl for the first time, demonstrating its outstanding performance.

Confocal structured illumination microscopy for improving the signal-to-noise ratio and depth of fluorescent optical section imaging

Light scattering from the sample is an unavoidable problem in fluorescence imaging. Compared with laser scanning confocal scanning microscopy, although optical-sectioning structured illumination microscopy (OS-SIM) has the advantages of fast imaging speed and low phototoxicity, it faces the challenge of removing the scattering fluorescent noise particularly when imaging thick and densely labeled sampling. To improve the imaging performance of OS-SIM, we introduce the concept of confocal imaging to OS-SIM and propose confocal structured illumination microscopy (CSIM). CSIM exploits the principle of dual imaging to reconstruct a dual image from each camera pixel. The scattered fluorescent noise and the unscattered fluorescent signal recorded by the camera pixel are separated in the reconstructed dual image. By extracting the unscattered fluorescent signal from each dual image based on the conjugate relationship between the camera and the spatial light modulator, we can eliminate the scattered fluorescent noise and reconstruct a confocal image. We have built the theoretical framework of CSIM. Experimental results of fluorescent optical-sectioning demonstrate that CSIM achieves a superior performance in eliminating scattered fluorescent noise and in extending imaging depth compared with existing OS-SIM. CSIM is expected to broaden the application range of OS-SIM.

The proximal centriole-like structure maintains nucleus-centriole architecture in sperm.

Proper connection between the sperm head and tail is critical for sperm motility and fertilization. Head-tail linkage is mediated by the head-tail coupling apparatus (HTCA), which secures the axoneme (tail) to the nucleus (head). However, the molecular architecture of the HTCA is poorly understood. Here, we use Drosophila to investigate formation and remodeling of the HTCA throughout spermiogenesis by visualizing key components of this complex. Using structured illumination microscopy, we demonstrate that key HTCA proteins Spag4 and Yuri form a 'centriole cap' that surrounds the centriole (or basal body) as it invaginates into the surface of the nucleus. As development progresses, the centriole is laterally displaced to the side of the nucleus while the HTCA expands under the nucleus, forming what we term the 'nuclear shelf'. We next show that the proximal centriole-like (PCL) structure is positioned under the nuclear shelf, functioning as a crucial stabilizer of centriole-nucleus attachment. Together, our data indicate that the HTCA is a complex, multi-point attachment site that simultaneously engages the PCL, the centriole and the nucleus to ensure proper head-tail connection during late-stage spermiogenesis.

A Denoise Network for Structured Illumination Microscopy with Low-Light Exposure

Super-resolution structured illumination microscopy (SR-SIM) is one of the important techniques that are most suitable for live-cell imaging. The reconstructed SR-SIM images are noisy once the raw images are recorded with low-light exposure. Here, we propose a new network (entitled the ND-SIM network) to denoise the SR images reconstructed using frequency-domain algorithms (FDAs). We demonstrate that ND-SIM can yield artifact-free SR images using raw images with an average photon count down to 20 per pixel while achieving comparable resolution to the ground truth (GT) obtained with high-light exposure. We can envisage that the ND-SIM will be widely applied for the long-term, super-resolution live-cell imaging of various bioprocesses in the future.

Super-Resolution Imaging Reveals the Mechanism of Endosomal Acidification Inhibitors Against SARS-CoV-2 Infection.

In this study, super-resolution structured illumination microscope (SIM) was used to analyze molecular mechanism of endocytic acidification inhibitors in the SARS-CoV-2 pandemic, such as Chloroquine (CQ), Hydroxychloroquine (HCQ) and Bafilomycin A1 (BafA1). We fluorescently labeled the SARS-CoV-2 RBD and its receptor ACE2 protein with small molecule dyes. Utilizing SIM imaging, the real-time impact of inhibitors (BafA1, CQ, HCQ, Dynasore) on the RBD-ACE2 endocytotic process was dynamically tracked in living cells. Initially, the protein activity of RBD and ACE2 was ensured after being labeled. And then our findings revealed that these inhibitors could inhibit the internalization and degradation of RBD-ACE2 to varying degrees. Among them, 100 nM BafA1 exhibited the most satisfactory endocytotic inhibition (~63.9 %) and protein degradation inhibition (~97.7 %). And it could inhibit the fusion between endocytic vesicles in the living cells. Additionally, Dynasore, a widely recognized dynein inhibitor, also demonstrated cell acidification inhibition effects. Together, these inhibitors collectively hinder SARS-CoV-2 infection by inhibiting both the viral internalization and RNA release. The comprehensive evaluation of pharmacological mechanisms through super-resolution fluorescence imaging has laid a crucial theoretical foundation for the development of potential drugs to treat COVID-19.