Structured Illumination Microscopy Data Research Articles

Approaching Maximum Resolution in Structured Illumination Microscopy via Accurate Noise Modeling.

Biological images captured by microscopes are characterized by heterogeneous signal-to-noise ratios (SNRs) due to spatially varying photon emission across the field of view convoluted with camera noise. State-of-the-art unsupervised structured illumination microscopy (SIM) reconstruction algorithms, commonly implemented in the Fourier domain, do not accurately model this noise and suffer from high-frequency artifacts, user-dependent choices of smoothness constraints making assumptions on biological features, and unphysical negative values in the recovered fluorescence intensity map. On the other hand, supervised methods rely on large datasets for training, and often require retraining for new sample structures. Consequently, achieving high contrast near the maximum theoretical resolution in an unsupervised, physically principled, manner remains an open problem. Here, we propose Bayesian-SIM (B-SIM), an unsupervised Bayesian framework to quantitatively reconstruct SIM data, rectifying these shortcomings by accurately incorporating known noise sources in the spatial domain. To accelerate the reconstruction process, we use the finite extent of the point-spread-function to devise a parallelized Monte Carlo strategy involving chunking and restitching of the inferred fluorescence intensity. We benchmark our framework on both simulated and experimental images, and demonstrate improved contrast permitting feature recovery at up to 25% shorter length scales over state-of-the-art methods at both high- and low-SNR. B-SIM enables unsupervised, quantitative, physically accurate reconstruction without the need for labeled training data, democratizing high-quality SIM reconstruction and expands the capabilities of live-cell SIM to lower SNR, potentially revealing biological features in previously inaccessible regimes.

Evaluation of Swin Transformer and knowledge transfer for denoising of super-resolution structured illumination microscopy data.

Convolutional neural network (CNN)-based methods have shown excellent performance in denoising and reconstruction of super-resolved structured illumination microscopy (SR-SIM) data. Therefore, CNN-based architectures have been the focus of existing studies. However, Swin Transformer, an alternative and recently proposed deep learning-based image restoration architecture, has not been fully investigated for denoising SR-SIM images. Furthermore, it has not been fully explored how well transfer learning strategies work for denoising SR-SIM images with different noise characteristics and recorded cell structures for these different types of deep learning-based methods. Currently, the scarcity of publicly available SR-SIM datasets limits the exploration of the performance and generalization capabilities of deep learning methods. In this work, we present SwinT-fairSIM, a novel method based on the Swin Transformer for restoring SR-SIM images with a low signal-to-noise ratio. The experimental results show that SwinT-fairSIM outperforms previous CNN-based denoising methods. Furthermore, as a second contribution, two types of transfer learning-namely, direct transfer and fine-tuning-were benchmarked in combination with SwinT-fairSIM and CNN-based methods for denoising SR-SIM data. Direct transfer did not prove to be a viable strategy, but fine-tuning produced results comparable to conventional training from scratch while saving computational time and potentially reducing the amount of training data required. As a third contribution, we publish four datasets of raw SIM images and already reconstructed SR-SIM images. These datasets cover two different types of cell structures, tubulin filaments and vesicle structures. Different noise levels are available for the tubulin filaments. The SwinT-fairSIM method is well suited for denoising SR-SIM images. By fine-tuning, already trained models can be easily adapted to different noise characteristics and cell structures. Furthermore, the provided datasets are structured in a way that the research community can readily use them for research on denoising, super-resolution, and transfer learning strategies.

Three-dimensional structured illumination microscopy data of mitochondria and lysosomes in cardiomyoblasts under normal and galactose-adapted conditions

This three-dimensional structured illumination microscopy (3DSIM) dataset was generated to highlight the suitability of 3DSIM to investigate mitochondria-derived vesicles (MDVs) in H9c2 cardiomyoblasts in living or fixed cells. MDVs act as a mitochondria quality control mechanism. The cells were stably expressing the tandem-tag eGFP-mCherry-OMP25-TM (outer mitochondrial membrane) which can be used as a sensor for acidity. A part of the dataset is showing correlative imaging of lysosomes labeled using LysoTracker in fixed and living cells. The cells were cultivated in either normal or glucose-deprived medium containing galactose. The resulting 3DSIM data were of high quality and can be used to undertake a variety of studies. Interestingly, many dynamic tubules derived from mitochondria are visible in the 3DSIM videos under both glucose and galactose-adapted growth conditions. As the raw 3DSIM data, optical parameters, and reconstructed 3DSIM images are provided, the data is especially suitable for use in the development of SIM reconstruction algorithms, bioimage analysis methods, and for biological studies of mitochondria.

Answering some questions about structured illumination microscopy.

Structured illumination microscopy (SIM) provides images of fluorescent objects at an enhanced resolution greater than that of conventional epifluorescence wide-field microscopy. Initially demonstrated in 1999 to enhance the lateral resolution twofold, it has since been extended to enhance axial resolution twofold (2008), applied to live-cell imaging (2009) and combined with myriad other techniques, including interferometric detection (2008), confocal microscopy (2010) and light sheet illumination (2012). Despite these impressive developments, SIM remains, perhaps, the most poorly understood ‘super-resolution’ method. In this article, we provide answers to the 13 questions regarding SIM proposed by Prakash et al. along with answers to a further three questions. After providing a general overview of the technique and its developments, we explain why SIM as normally used is still diffraction-limited. We then highlight the necessity for a non-polynomial, and not just nonlinear, response to the illuminating light in order to make SIM a true, diffraction-unlimited, super-resolution technique. In addition, we present a derivation of a real-space SIM reconstruction approach that can be used to process conventional SIM and image scanning microscopy (ISM) data and extended to process data with quasi-arbitrary illumination patterns. Finally, we provide a simple bibliometric analysis of SIM development over the past two decades and provide a short outlook on potential future work.This article is part of the Theo Murphy meeting issue ‘Super-resolution structured illumination microscopy (part 2)’.

Polarized illumination coded structured illumination microscopy (picoSIM): experimental results.

The need for acquiring at least three images to reconstruct an optical section of a sample limits the acquisition rate in structured illumination microscopy (SIM) for optical sectioning. In polarized illumination coded structured illumination microscopy (picoSIM) the three individual light patterns are encoded in a single polarized illumination light distribution, enabling the acquisition of the complete SIM data in a single exposure. Here, we describe our experimental set-up and show experimental results acquired with sequential and single-shot picoSIM. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 2)'.

Super-resolution imaging illuminates new dynamic behaviors of cellulose synthase.

Confocal imaging has shown that CELLULOSE SYNTHASE (CESA) particles move through the plasma membrane as they synthesize cellulose. However, the resolution limit of confocal microscopy circumscribes what can be discovered about these tiny biosynthetic machines. Here, we applied Structured Illumination Microscopy (SIM), which improves resolution two-fold over confocal or widefield imaging, to explore the dynamic behaviors of CESA particles in living plant cells. SIM imaging reveals that Arabidopsis thaliana CESA particles are more than twice as dense in the plasma membrane as previously estimated, helping explain the dense arrangement of cellulose observed in new wall layers. CESA particles tracked by SIM display minimal variation in velocity, suggesting coordinated control of CESA catalytic activity within single complexes and that CESA complexes might move steadily in tandem to generate larger cellulose fibrils or bundles. SIM data also reveal that CESA particles vary in their overlaps with microtubule tracks and can complete U-turns without changing speed. CESA track patterns can vary widely between neighboring cells of similar shape, implying that cellulose patterning is not the sole determinant of cellular growth anisotropy. Together, these findings highlight SIM as a powerful tool to advance CESA imaging beyond the resolution limit of conventional light microscopy.

Sim-Ptychography Imaging of Hutchinson-Gilford Progeria Syndrome (HGPS) Cells

Hutchinson-Gilford progeria syndrome (HGPS) is a genetic condition characterized by the rapid appearance of aging beginning in childhood. HGPS is caused by a mutation in the encoding gene of lamin-A, one important component of the nuclear lamina. Lamins provide shape and structural stability to the nucleus, and are involved in many cellular processes (DNA replication and repair, chromatin organization, etc). In HGPS the mutation of lamin-A results in a distorted nuclear lamina. It is crucial to understand if this mutation affects only the structure of the nucleus, or if other alterations occur. Optical microscopy is one of the most used tools to investigate biological processes, with high-resolution imaging commonly performed using fluorophores. However, live-cell imaging is not always compatible with labeling due to photobleaching and phototoxicity effects. Label-free imaging is therefore of great interest to study biological systems. One such method is ptychography, a computational approach able to retrieve the phase and amplitude of a sample by analyzing collected diffraction patterns. These phase maps contain information about thickness and refractive index, and allow quantification of cellular morphology. However, its limitation is the comparatively low achievable resolution. Previous work has shown the advantage of using structured illumination together with ptychography to increase the spatial resolution. For this reason, we couple ptychography with structured illumination microscopy (SIM) to perform high-resolution imaging. In this work, a ptychographic module was developed and mounted on a commercial SIM, to simultaneously collect ptychography and SIM data. We show how changes in the morphology of a HGPS nucleus affect the phase compared to normal cells, and how the SIM-ptychography approach can increase the resolution compared with conventional optical ptychography.

Enhanced reconstruction of structured illumination microscopy on a polarized specimen.

Structured illumination microscopy (SIM) requires polarization control to guarantee the high-contrast illumination pattern. However, this modulated polarization will induce artifacts in SIM when imaging fluorescent dipoles. Here we proposed the polarization weighted recombination of frequency components to reconstruct SIM data with suppressed artifacts and better resolving power. Both the simulation results and experimental data demonstrate that our algorithm can obtain isotropic resolution on dipoles and resolve a clearer structure in high-density sections compared to the conventional algorithm. Our work reinforces the SIM theory and paves the avenue for the application of SIM on a polarized specimen.

Super-Resolution Structured Illumination Microscopy Reconstruction Using a Least-Squares Solver

Super-resolution microscopy enables images to be obtained at a resolution higher than that imposed by the diffraction limit of light. Structured illumination microscopy (SIM) is among the fastest super-resolution microscopy techniques currently in use, and it has gained popularity in the field of cytobiology research owing to its low photo-toxicity and widefield modality. In typical SIM, a fluorescent sample is excited by sinusoidal patterns by employing a linear strategy to reconstruct super-resolution images. However, this strategy fails in cases where non-sinusoidal illumination patterns are used. In this study, we propose the least-squares SIM (LSQ-SIM) approach, which is an efficient super-resolution reconstruction algorithm in the framework of least-squares regression that can process raw SIM data under both sinusoidal and non-sinusoidal illuminations. The results obtained in this study indicate the potential of LSQ-SIM for use in structured illumination microscopy and its various application fields.

Fast reconstruction algorithm for structured illumination microscopy.

Structured illumination microscopy (SIM) is a powerful technique for providing super-resolution imaging, but its reconstruction algorithm, i.e., linear reconstruction structured illumination microscopy (LRSIM) algorithm in the Fourier domain, limits the imaging speed due to its computational effort. Here, we present a novel reconstruction algorithm that can directly process SIM data in the spatial domain. Compared to LRSIM, this approach uses the same number of frames to achieve a comparable resolution but with a much faster processing speed. Our algorithm was verified on both simulated and experimental data using sinusoidal pattern illumination. Moreover, this algorithm is also applicable for speckle pattern illumination.

A method for the reconstruction of multifocal structured illumination microscopy data with high efficiency

We present and demonstrate an efficient method for the reconstruction of profiles acquired by multifocal structured illumination microscopy (MSIM) utilizing few raw images. Firstly, we propose a method to produce nine raw multifocal images with enhanced modulation depth to accomplish the uniform illumination of the sample. Then, combing with the parameter of the arrays, we perform the standard construct reconstruction procedure of structured illumination microscopy (SIM) row by row and column by column. Finally, we combine these restored images together to obtain the final image with enhanced resolution and good contrast. Based on theoretical analysis and numerical simulations, this method shows great potential in the field of the image reconstruction of MSIM data.

A Novel Viewpoint to Analyze Structured Illumination Microscopy (Sim) Data

Structured Illumination Microscopy (SIM) is an important tool among the Super Resolution Microscopy techniques [1]. SIM stands out for its capability to perform optical sectioning, multicolor channel acquisition and live cell imaging, thanks to its low phototoxicity. In SIM, the specimen is illuminated with a high-frequency grating pattern, instead of a uniform field. The multiplication of sample features and this pattern gives rise to Moiré fringes.The obtained interference pattern allows encoding of sub-diffraction sample's features into lower frequencies collected by the objective lens. The reconstruction process then decodes otherwise unresolvable high-frequency information creating the super-resolved image.Here we aim to analyze SIM data from a rather different standpoint, based on the idea of Separation of Photons by LIfetime Tuning (SPLIT)[2]. In SPLIT the sub-diffraction spatial information is encoded into an additional channel of the microscope. The information encoded within this channel is visualized and analyzed in the phasor plot and allows separating a fractional image component of higher spatial resolution corresponding to the center of the PSF.In the context of Stimulated Emission Depletion (STED) microscopy, we recently demonstrated that SPLIT can be used to decode spatial information encoded in the nanosecond fluorescence lifetime [2] or variations of the depletion power [3]. We show here that in the case of SIM it is possible to use SPLIT to analyze the information encoded into the images acquired at different illumination patterns. In particular an important aspect of SPLIT is that the Phasor Plot provides a visual, intuitive and direct evaluation of the acquired data [3].We demonstrate that knowledge of the illumination pattern can be used to perform SPLIT analysis and to reconstruct a super-resolved image bypassing the traditional Fourier reconstruction.As an application, we use the SPLIT-SIM approach to perform super-resolution imaging of chromatin-related structures.

Imaging tissues and cells beyond the diffraction limit with structured illumination microscopy and Bayesian image reconstruction

BackgroundStructured illumination microscopy (SIM) is a family of methods in optical fluorescence microscopy that can achieve both optical sectioning and super-resolution effects. SIM is a valuable method for high-resolution imaging of fixed cells or tissues labeled with conventional fluorophores, as well as for imaging the dynamics of live cells expressing fluorescent protein constructs. In SIM, one acquires a set of images with shifting illumination patterns. This set of images is subsequently treated with image analysis algorithms to produce an image with reduced out-of-focus light (optical sectioning) and/or with improved resolution (super-resolution).FindingsFive complete, freely available SIM datasets are presented including raw and analyzed data. We report methods for image acquisition and analysis using open-source software along with examples of the resulting images when processed with different methods. We processed the data using established optical sectioning SIM and super-resolution SIM methods and with newer Bayesian restoration approaches that we are developing.ConclusionsVarious methods for SIM data acquisition and processing are actively being developed, but complete raw data from SIM experiments are not typically published. Publically available, high-quality raw data with examples of processed results will aid researchers when developing new methods in SIM. Biologists will also find interest in the high-resolution images of animal tissues and cells we acquired. All of the data were processed with SIMToolbox, an open-source and freely available software solution for SIM.

Image restoration approach to address reduced modulation contrast in structured illumination microscopy.

The performance of structured illumination microscopy (SIM) is hampered in many biological applications due to the inability to modulate the light when imaging deep into the sample. This is in part because sample-induced aberration reduces the modulation contrast of the structured pattern. In this paper, we present an image restoration approach suitable for processing raw incoherent-grid-projection SIM data with a low fringe contrast. Restoration results from simulated and experimental ApoTome SIM data show results with improved signal-to-noise ratio (SNR) and optical sectioning compared to the results obtained from existing methods, such as 2D demodulation and 3D SIM deconvolution. Our proposed method provides satisfactory results (quantified by the achieved SNR and normalized mean square error) even when the modulation contrast of the illumination pattern is as low as 7%.

Chromatin organization at the nuclear periphery as revealed by image analysis of structured illumination microscopy data

The nuclear periphery (NP) plays a substantial role in chromatin organization. Heterochromatin at the NP is interspersed with active chromatin surrounding nuclear pore complexes (NPCs); however, details of the peripheral chromatin organization are missing. To discern the distribution of epigenetic marks at the NP of HeLa nuclei, we used structured illumination microscopy combined with a new MATLAB software tool for automatic NP and NPC detection, measurements of fluorescent intensity and statistical analysis of measured data. Our results show that marks for both active and non-active chromatin associate differentially with NPCs. The incidence of heterochromatin marks, such as H3K27me2 and H3K9me2, was significantly lower around NPCs. In contrast, the presence of marks of active chromatin such as H3K4me2 was only decreased very slightly around the NPCs or not at all (H3K9Ac). Interestingly, the histone demethylases LSD1 (also known as KDM1A) and KDM2A were enriched within the NPCs, suggesting that there was a chromatin-modifying mechanism at the NPCs. Inhibition of transcription resulted in a larger drop in the distribution of H1, H3K9me2 and H3K23me2, which implies that transcription has a role in the organization of heterochromatin at the NP.

Chromatin organization at the nuclear periphery as revealed by image analysis of structured illumination microscopy data.

The nuclear periphery (NP) plays a substantial role in chromatin organization. Heterochromatin at the NP is interspersed with active chromatin surrounding nuclear pore complexes (NPCs); however, details of the peripheral chromatin organization are missing. To discern the distribution of epigenetic marks at the NP of HeLa nuclei, we used structured illumination microscopy combined with a new MATLAB software tool for automatic NP and NPC detection, measurements of fluorescent intensity and statistical analysis of measured data. Our results show that marks for both active and non-active chromatin associate differentially with NPCs. The incidence of heterochromatin marks, such as H3K27me2 and H3K9me2, was significantly lower around NPCs. In contrast, the presence of marks of active chromatin such as H3K4me2 was only decreased very slightly around the NPCs or not at all (H3K9Ac). Interestingly, the histone demethylases LSD1 (also known as KDM1A) and KDM2A were enriched within the NPCs, suggesting that there was a chromatin-modifying mechanism at the NPCs. Inhibition of transcription resulted in a larger drop in the distribution of H1, H3K9me2 and H3K23me2, which implies that transcription has a role in the organization of heterochromatin at the NP.

Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ.

Super-resolved structured illumination microscopy (SR-SIM) is an important tool for fluorescence microscopy. SR-SIM microscopes perform multiple image acquisitions with varying illumination patterns, and reconstruct them to a super-resolved image. In its most frequent, linear implementation, SR-SIM doubles the spatial resolution. The reconstruction is performed numerically on the acquired wide-field image data, and thus relies on a software implementation of specific SR-SIM image reconstruction algorithms. We present fairSIM, an easy-to-use plugin that provides SR-SIM reconstructions for a wide range of SR-SIM platforms directly within ImageJ. For research groups developing their own implementations of super-resolution structured illumination microscopy, fairSIM takes away the hurdle of generating yet another implementation of the reconstruction algorithm. For users of commercial microscopes, it offers an additional, in-depth analysis option for their data independent of specific operating systems. As a modular, open-source solution, fairSIM can easily be adapted, automated and extended as the field of SR-SIM progresses.

Cortical Actin Flow in T Cells Quantified by Spatio-temporal Image Correlation Spectroscopy of Structured Illumination Microscopy Data.

Filamentous-actin plays a crucial role in a majority of cell processes including motility and, in immune cells, the formation of a key cell-cell interaction known as the immunological synapse. F-actin is also speculated to play a role in regulating molecular distributions at the membrane of cells including sub-membranous vesicle dynamics and protein clustering. While standard light microscope techniques allow generalized and diffraction-limited observations to be made, many cellular and molecular events including clustering and molecular flow occur in populations at length-scales far below the resolving power of standard light microscopy. By combining total internal reflection fluorescence with the super resolution imaging method structured illumination microscopy, the two-dimensional molecular flow of F-actin at the immune synapse of T cells was recorded. Spatio-temporal image correlation spectroscopy (STICS) was then applied, which generates quantifiable results in the form of velocity histograms and vector maps representing flow directionality and magnitude. This protocol describes the combination of super-resolution imaging and STICS techniques to generate flow vectors at sub-diffraction levels of detail. This technique was used to confirm an actin flow that is symmetrically retrograde and centripetal throughout the periphery of T cells upon synapse formation.

Cortical Actin Flow in T Cells Quantified by Spatio-temporal Image Correlation Spectroscopy of Structured Illumination Microscopy Data

Filamentous-actin plays a crucial role in a majority of cell processes including motility and, in immune cells, the formation of a key cell-cell interaction known as the immunological synapse. F-actin is also speculated to play a role in regulating molecular distributions at the membrane of cells including sub-membranous vesicle dynamics and protein clustering. While standard light microscope techniques allow generalized and diffraction-limited observations to be made, many cellular and molecular events including clustering and molecular flow occur in populations at length-scales far below the resolving power of standard light microscopy. By combining total internal reflection fluorescence with the super resolution imaging method structured illumination microscopy, the two-dimensional molecular flow of F-actin at the immune synapse of T cells was recorded. Spatio-temporal image correlation spectroscopy (STICS) was then applied, which generates quantifiable results in the form of velocity histograms and vector maps representing flow directionality and magnitude. This protocol describes the combination of super-resolution imaging and STICS techniques to generate flow vectors at sub-diffraction levels of detail. This technique was used to confirm an actin flow that is symmetrically retrograde and centripetal throughout the periphery of T cells upon synapse formation.

In situ structure of FtsZ mini-rings in Arabidopsis chloroplasts

AbstractChloroplasts are essential plant organelles that divide by binary fission through a coordinated ring-shaped division machinery located both on the outside and inside of the chloroplast. The first step in chloroplast division is the assembly of an internal division ring (Z-ring) that is composed of the key filamentous chloroplast division proteins FtsZ1 and FtsZ2. How the individual FtsZ filaments assemble into higher-order structures to form the dividing Z-ring is not well understood and the most detailed insights have so far been gleaned from prokaryotic FtsZ. Here, we present in situ data of chloroplast FtsZ making use of a smaller ring-like FtsZ assembly termed mini-rings that form under well-defined conditions. Structured illumination microscopy (SIM) permitted their mean diameter to be determined as 208 nm and also showed that 68 % of these rings are terminally attached to linear FtsZ filaments. A correlative microscopy-compatible specimen preparation based on freeze substitution after high-pressure freezing is presented addressing the challenges such as autofluorescence and specific fluorescence attenuation. Transmission electron microscopy (TEM) and scanning TEM (STEM) imaging of thin sections exhibited ring-like densities that matched in size with the SIM data, and TEM tomography revealed insights into the molecular architecture of mini-rings demonstrating the following key features: (1) overall, a roughly bipartite split into a more ordered/curved and less ordered/curved half is readily discernible; (2) the density distribution in individual strands matches with the X-ray data, suggesting they constitute FtsZ protofilaments; (3) in the less ordered half of the ring, the protofilaments are able to assemble into higher-order structures such as double helices and supercoiled structures. Taken together, the data suggest that the state of existence of mini-rings could be described as metastable and their possible involvement in filament storage and Z-ring assembly is discussed.