Conventional Structured Illumination Microscopy Research Articles

Large-field and fast super-resolution microscopic imaging method based on laser interferometry

In recent years, structured illumination microscopy (SIM) has been drawing great attention for both technique development and application. However, conventional SIM, which uses a spatial light modulator (SLM) for fringe projection, often has a limited field of view. To meet the demand for high-throughput microscopic imaging in biomedicine research, a large-field super-resolution (SR) fluorescence microscopic imaging method based on laser interferometry was proposed. The method that combines a two-dimensional (2D) grating for fringe pattern projection and an SLM for selecting fringe orientation can break the limitation of fringe number limited by the digital projection devices. A spatial-domain reconstruction algorithm was developed to improve the computational speed of super-resolution imaging. Finally, an experimental platform for SIM microscopy was established. A large-field view of 1380 μm × 1035 μm under a 20×/NA0.75 objective is experimentally demonstrated, and an enhancement of 1.8-fold resolution is realized. The spatial-domain reconstruction algorithm can significantly improve the computational speed by approximately 10 times faster compared to the traditional frequency-domain algorithm.

Faster structured illumination microscopy using complementary encoding-based compressive imaging

Structured illumination microscopy (SIM) has been widely applied to investigate intricate biological dynamics due to its outstanding super-resolution imaging speed. Incorporating compressive sensing into SIM brings the possibility to further improve the super-resolution imaging speed. Nevertheless, the recovery of the super-resolution information from the compressed measurement remains challenging in experiments. Here, we report structured illumination microscopy with complementary encoding-based compressive imaging (CECI-SIM) to realize faster super-resolution imaging. Compared to the nine measurements to obtain a super-resolution image in a conventional SIM, CECI-SIM can achieve a super-resolution image by three measurements; therefore, a threefold improvement in the imaging speed can be achieved. This faster imaging ability in CECI-SIM is experimentally verified by observing tubulin and actin in mouse embryonic fibroblast cells. This work provides a feasible solution for high-speed super-resolution imaging, which would bring significant applications in biomedical research.

Structured illumination microscopy with a phase-modulated spinning disk for optical sectioning.

Among various super-resolution microscopic techniques, structured illumination microscopy (SIM) stands out for live-cell imaging because of its higher imaging speed. However, conventional SIM lacks optical sectioning capability. Here we demonstrate a new, to the best of our knowledge, approach using a phase-modulated spinning disk (PMSD) that enhances the optical sectioning capability of SIM. The PMSD consists of a pinhole array for confocal imaging and a transparent polymer layer for light phase modulation. The light phase modulation was designed to cancel the zeroth-order diffracted beam and create a sharp lattice illumination pattern using the interference of four first-order diffracted beams. In the detection optical path, the PMSD serves as a spatial filter to physically reject about 80% of the out-of-focus signals, an approach that allows for real-time optical reconstruction of super-resolved images with enhanced contrast. Furthermore, the simplicity of the design makes it easy to upgrade a conventional fluorescence microscope to a PMSD SIM system.

Deep neural network based on multi-level wavelet and attention for structured illumination microscopy

Structured illumination microscopy (SIM) is a popular and powerful super-resolution (SR) technique in biomedical research. However, the conventional reconstruction algorithm for SIM heavily relies on the accurate prior knowledge of illumination patterns and signal-to-noise ratio (SNR) of raw images. To obtain high-quality SR images, several raw images need to be captured under high fluorescence level, which further restricts SIM’s temporal resolution and its applications. Deep learning (DL) is a data-driven technology that has been used to expand the limits of optical microscopy. In this study, we propose a deep neural network based on multi-level wavelet and attention mechanism (MWAM) for SIM. Our results show that the MWAM network can extract high-frequency information contained in SIM raw images and accurately integrate it into the output image, resulting in superior SR images compared to those generated using wide-field images as input data. We also demonstrate that the number of SIM raw images can be reduced to three, with one image in each illumination orientation, to achieve the optimal tradeoff between temporal and spatial resolution. Furthermore, our MWAM network exhibits superior reconstruction ability on low-SNR images compared to conventional SIM algorithms. We have also analyzed the adaptability of this network on other biological samples and successfully applied the pretrained model to other SIM systems.

Resolution Improvement of Structured Illumination Microscopy using Non‐Diffraction‐Limited Point‐Scanned Fluorescence Fringe

AbstractTo further improve the resolving ability of structured illumination microscopy (SIM), a space‐time‐modulation–based laser‐scanning structured illumination microscopy (LSSIM) technique is presented in which a non‐diffraction‐limited non‐sinusoidal cumulative fluorescence fringe is generated in samples by point scanning the modulated excitation beam. The higher‐frequency information of samples can be mixed using the cumulative fringes with unlimited frequency into the passband of a confocal system. The non‐sinusoidal intensity‐modulated function used in LSSIM is specifically designed as a combination of cosine functions with different orders. The resolving ability of LSSIM experimentally by imaging fluorescent beads and bovine pulmonary artery endothelial cells with stained F‐actin is demonstrated. The results demonstrate three‐fold resolution enhancement over the confocal system, surpassing the resolving ability of conventional SIM. Moreover, the optical system is a simple modification of laser‐scanning confocal microscopy, making LSSIM more compatible with other techniques and suitable for biomedical research without the specific requirements for fluorescent dyes.

High-speed spatially re-modulated structured illumination microscopy.

Structured illumination microscopy (SIM) allows non-invasive visualization of nanoscale subcellular structures. However, image acquisition and reconstruction become the bottleneck to further improve the imaging speed. Here, we propose a method to accelerate SIM imaging by combining the spatial re-modulation principle with Fourier domain filtering and using measured illumination patterns. This approach enables high-speed, high-quality imaging of dense subcellular structures using a conventional nine-frame SIM modality without phase estimation of the patterns. In addition, seven-frame SIM reconstruction and additional hardware acceleration further improve the imaging speed using our method. Furthermore, our method is also applicable to other spatially uncorrelated illumination patterns, such as distorted sinusoidal, multifocal, and speckle patterns.

Reconstruction of structured illumination microscopy with an untrained neural network

Structured illumination microscopy (SIM) is one of super-resolution optical microscopic techniques, and it has been widely used in biological research. In this paper, a physics-driven deep image prior framework for super-resolution reconstruction of SIM (entitled DIP-SIM) is proposed. DIP-SIM does not rely on a large number of labeled data, and the output becomes more interpretable due to the intrinsic constraint of a physical model. Both the simulation and experiment verify that DIP-SIM can reconstruct a super-resolution image with a quality comparable to conventional SIM. Of note, it allows for super-resolution reconstruction from three raw images for two-orientation SIM and four raw images for three-orientation SIM, and hence it has a much faster imaging speed and lower photobleaching compared with the traditional SIM. We can envisage that the proposed method can be applied to chemistry and biomedical fields, etc.

Hybrid reconstruction of the physical model with the deep learning that improves structured illumination microscopy

Structured illumination microscopy (SIM) has been widely used in live-cell superresolution (SR) imaging. However, conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw images with low signal-to-noise ratios (SNRs). Deep-learning (DL)-based methods can address this challenge but may lead to degradation and hallucinations. By combining the physical inversion model with a total deep variation (TDV) regularization, we propose a hybrid restoration method (TDV-SIM) that outperforms conventional or DL methods in suppressing artifacts and hallucinations while maintaining resolutions. We demonstrate the performance superiority of TDV-SIM in restoring actin filaments, endoplasmic reticulum, and mitochondrial cristae from extremely low SNR raw images. Thus TDV-SIM represents the ideal method for prolonged live-cell SR imaging with minimal exposure and photodamage. Overall, TDV-SIM proves the power of integrating model-based reconstruction methods with DL ones, possibly leading to the rapid exploration of similar strategies in high-fidelity reconstructions of other microscopy methods.

Confocal rescan structured illumination microscopy for real-time deep tissue imaging with superresolution

Structured illumination microscopy (SIM) is an established optical superresolution imaging technique. However, conventional SIM based on wide-field image acquisition is generally limited to visualizing thin cellular samples. We propose combining one-dimensional image rescan and structured illumination in the orthogonal direction to achieve superresolution without the need to rotate the illumination pattern. The image acquisition speed is consequently improved threefold, which is also beneficial for minimizing photobleaching and phototoxicity. Optical sectioning in thick biological tissue is enhanced by including a confocal slit in the system to significantly suppress the out-of-focus background and the associated noise. With all the technical improvements, our method captures three-dimensional superresolved image stacks of neuronal structures in mouse brain tissue samples for a depth range of more than 200 μm.

Surpassing the resolution limitation of structured illumination microscopy by an untrained neural network.

Structured illumination microscopy (SIM), as a flexible tool, has been widely applied to observing subcellular dynamics in live cells. It is noted, however, that SIM still encounters a problem with theoretical resolution limitation being only twice over wide-field microscopy, where imaging of finer biological structures and dynamics are significantly constrained. To surpass the resolution limitation of SIM, we developed an image postprocessing method to further improve the lateral resolution of SIM by an untrained neural network, i.e., deep resolution-enhanced SIM (DRE-SIM). DRE-SIM can further extend the spatial frequency components of SIM by employing the implicit priors based on the neural network without training datasets. The further super-resolution capability of DRE-SIM is verified by theoretical simulations as well as experimental measurements. Our experimental results show that DRE-SIM can achieve the resolution enhancement by a factor of about 1.4 compared with conventional SIM. Given the advantages of improving the lateral resolution while keeping the imaging speed, DRE-SIM will have a wide range of applications in biomedical imaging, especially when high-speed imaging mechanisms are integrated into the conventional SIM system.

Robust frame-reduced structured illumination microscopy with accelerated correlation-enabled parameter estimation

Structured illumination microscopy (SIM), with the advantages of full-field imaging and low photo-damage, is one of the most well-established fluorescence super-resolution microscopy techniques that raised great interest in biological sciences. However, conventional SIM techniques generally require at least nine images for image reconstruction, and the quality of super-resolution significantly depends on high-accuracy illumination parameter estimation, which is usually computationally intense and time-consuming. To address these issues, we propose a robust seven-frame SIM reconstruction algorithm with accelerated correlation-enabled parameter estimation. First, a modulation-assigned spatial filter is employed to remove unreliable backgrounds associated with low signal-to-noise ratios. Then, we propose a coarse-to-fine accelerated correlation algorithm to eliminate the redundant iterations of the traditional correlation-based scheme. The frame reduction is achieved by a specially designed phase-shifting strategy combined with pixel-wise fluorescence pre-calibration. We experimentally demonstrate that, compared with conventional iterative correlation-based methods, the proposed algorithm improves the computational efficiency by a factor of 4.5 while maintaining high accuracy illumination parameter estimation. Meanwhile, our method achieves high-quality super-resolution reconstruction even with a reduction in two raw images, which improves the efficiency of image acquisition and ensures the robustness toward complex experimental environments.

Background inhibited and speed-loss-free volumetric imaging in vivo based on structured-illumination Fourier light field microscopy.

Benefiting from its advantages in fast volumetric imaging for recording biodynamics, Fourier light field microscopy (FLFM) has a wide range of applications in biomedical research, especially in neuroscience. However, the imaging quality of the FLFM is always deteriorated by both the out-of-focus background and the strong scattering in biological samples. Here we propose a structured-illumination and interleaved-reconstruction based Fourier light field microscopy (SI-FLFM), in which we can filter out the background fluorescence in FLFM without sacrificing imaging speed. We demonstrate the superiority of our SI-FLFM in high-speed, background-inhibited volumetric imaging of various biodynamics in larval zebrafish and mice in vivo. The signal-to-background ratio (SBR) is improved by tens of times. And the volumetric imaging speed can be up to 40 Hz, avoiding artifacts caused by temporal under-sampling in conventional structured illumination microscopy. These suggest that our SI-FLFM is suitable for applications of weak fluorescence signals but high imaging speed requirements.

Hyperbolic metamaterial-assisted structured illumination microscopy using periodic sub-diffraction speckles

Structured illumination microscopy (SIM), as a wide-field, rapid, super-resolution imaging technology, is widely employed in the field of biology. In this work, we propose a hyperbolic metamaterial (HMM)-assisted super-resolution structured illumination microscopy technique. By utilizing the HMM, a sub-diffraction illumination pattern of bulk plasmon polariton (BPP) with pure and higher spatial frequency replaces the conventional laser interference fringes, thus the imaging resolution of BPPSIM could surpass that of conventional SIM and reach 65 nm for HMM with 8-layers periodic structure. Moreover, an HMM with 10-layers aperiodic structure designed with particle swarm optimization was obtained, and BPPSIM could bring the imaging resolution down to 60 nm (1/9 of the fluorescence wavelength), which is a 3.3-fold improvement compared with the diffraction-limited image. This BPPSIM would provide a super-resolution, wide field of view, and good bio-compatibility approach in biological imaging.

Large-field structured illumination microscopy based on 2D grating and a spatial light modulator.

Structured illumination microscopy (SIM) has been widely used in biological research due to its merits of fast imaging speed, minimal invasiveness, super-resolution, and optical sectioning imaging capability. However, the conventional SIM that uses a spatial light modulator (SLM) for fringe projection often has a limited imaging field of view. Herein, we report a large-field SIM technique that combines a 2D grating for fringe pattern projection and an SLM for selecting fringe orientation and performing phase shifting digitally. The proposed SIM technique breaks the bottleneck of fringe number limited by the digital projection devices, while maintaining the advantage of high-speed (digital) phase shifting of conventional SIM. The method avoids the pixilation and dispersion effects of the SLMs. Finally, a 1.8-fold resolution enhancement in a large field of 690 × 517 µm2 under a 20×/NA0.75 objective is experimentally demonstrated. The proposed technique can be widely applied to biology, chemistry, and industry.

High-speed super-resolution imaging with compressive imaging-based structured illumination microscopy.

Structured illumination microscopy (SIM) has been widely applied to investigating fine structures of biological samples by breaking the optical diffraction limitation. So far, video-rate imaging has been obtained in SIM, but the imaging speed was still limited due to the reconstruction of a super-solution image through multi-sampling, which hindered the applications in high-speed biomedical imaging. To overcome this limitation, here we develop compressive imaging-based structured illumination microscopy (CISIM) by synergizing SIM and compressive sensing (CS). Compared with conventional SIM, CISIM can greatly improve the super-resolution imaging speed by extracting multiple super-resolution images from one compressed image. Based on CISIM, we successfully reconstruct the super-resolution images in biological dynamics, and analyze the effect factors of image reconstruction quality, which verify the feasibility of CISIM. CISIM paves a way for high-speed super-resolution imaging, which may bring technological breakthroughs and significant applications in biomedical imaging.

Resolution enhancement in random illumination microscopy using photon correlations.

Random illumination microscopy (RIM) could surpass the diffraction barrier in fluorescence microscopy by illuminating an object with unknown speckle patterns. It has been demonstrated that the resolution in RIM using second-order statistics is as good as that of conventional structured illumination microscopy (SIM) from the asymptotic point of view. Compared to classical SIM, RIM is more robust to optical aberrations and scattering introduced by thick samples. In this work, I show that the quantum correlations could further improve the resolution in random illumination microscopy due to the photon antibunching property of fluorophore emitters. In theory, the super-resolution capacity of this quantum-enhanced version of RIM corresponds to the fourth power of the point spread function under the general epi-illumination geometry.

Answering some questions about structured illumination microscopy.

Structured illumination microscopy (SIM) provides images of fluorescent objects at an enhanced resolution greater than that of conventional epifluorescence wide-field microscopy. Initially demonstrated in 1999 to enhance the lateral resolution twofold, it has since been extended to enhance axial resolution twofold (2008), applied to live-cell imaging (2009) and combined with myriad other techniques, including interferometric detection (2008), confocal microscopy (2010) and light sheet illumination (2012). Despite these impressive developments, SIM remains, perhaps, the most poorly understood ‘super-resolution’ method. In this article, we provide answers to the 13 questions regarding SIM proposed by Prakash et al. along with answers to a further three questions. After providing a general overview of the technique and its developments, we explain why SIM as normally used is still diffraction-limited. We then highlight the necessity for a non-polynomial, and not just nonlinear, response to the illuminating light in order to make SIM a true, diffraction-unlimited, super-resolution technique. In addition, we present a derivation of a real-space SIM reconstruction approach that can be used to process conventional SIM and image scanning microscopy (ISM) data and extended to process data with quasi-arbitrary illumination patterns. Finally, we provide a simple bibliometric analysis of SIM development over the past two decades and provide a short outlook on potential future work.This article is part of the Theo Murphy meeting issue ‘Super-resolution structured illumination microscopy (part 2)’.

Recent Progress in the Correlative Structured Illumination Microscopy

The super-resolution imaging technique of structured illumination microscopy (SIM) enables the mixing of high-frequency information into the optical transmission domain via light-source modulation, thus breaking the optical diffraction limit. Correlative SIM, which combines other techniques with SIM, offers more versatility or higher imaging resolution than traditional SIM. In this review, we first briefly introduce the imaging mechanism and development trends of conventional SIM. Then, the principles and recent developments of correlative SIM techniques are reviewed. Finally, the future development directions of SIM and its correlative microscopies are presented.

Image reconstruction approach for a high space-bandwidth product structured illumination microscopy system

Conventional structured illumination microscopy (SIM) utilizes a sinusoidal excitation pattern of frequency within the detection passband and provides a maximum of twofold resolution enhancement over the diffraction limit. A transmission approach proposed in an earlier publication [J. Phys. D 53, 044006 (2019)JPAPBE0022-372710.1088/1361-6463/ab4e68] to further improve the lateral resolution requires sequential higher frequency illumination patterns. However, the existing reconstruction algorithms fail to deliver appropriate reconstruction when the excitation frequency lies far from the passband boundary. Here, we present a correlation-based SIM reconstruction approach for sequential high-frequency illumination patterns even if the pattern frequency lies far from the passband limit. The scheme can be suitably implemented in a variety of custom-built systems where illumination frequency lies beyond the passband support (e.g., non-linear SIM and plasmonic SIM).

Inside Front Cover

AbstractWide‐field, optical‐sectioning imaging of neural network in the hippocampus of Thy1‐YFP mouse brain, with (up‐right) and without (down‐left) structured illumination. By interleaved reconstruction, we can recover the speed loss in conventional structured illumination microscopy, which makes it promising for large‐scale imaging of biodynamics at high spatio‐temporal resolutions.Further details can be found in the article by Ruheng Shi, Yuting Li, and Lingjie Kong (e202000513) image