3D Single-molecule Localization Microscopy Research Articles

Enhancing the data processing speed of a deep-learning-based three-dimensional single molecule localization algorithm (FD-DeepLoc) with a combination of feature compression and pipeline programming

Three-dimensional (3D) single molecule localization microscopy (SMLM) plays an important role in biomedical applications, but its data processing is very complicated. Deep learning is a potential tool to solve this problem. As the state of art 3D super-resolution localization algorithm based on deep learning, FD-DeepLoc algorithm reported recently still has a gap with the expected goal of online image processing, even though it has greatly improved the data processing throughput. In this paper, a new algorithm Lite-FD-DeepLoc is developed on the basis of FD-DeepLoc algorithm to meet the online image processing requirements of 3D SMLM. This new algorithm uses the feature compression method to reduce the parameters of the model, and combines it with pipeline programming to accelerate the inference process of the deep learning model. The simulated data processing results show that the image processing speed of Lite-FD-DeepLoc is about twice as fast as that of FD-DeepLoc with a slight decrease in localization accuracy, which can realize real-time processing of [Formula: see text] pixels size images. The results of biological experimental data processing imply that Lite-FD-DeepLoc can successfully analyze the data based on astigmatism and saddle point engineering, and the global resolution of the reconstructed image is equivalent to or even better than FD-DeepLoc algorithm.

High Refractive Index Imaging Buffer for Dual-Color 3D SMLM Imaging of Thick Samples.

The current limitations of single-molecule localization microscopy (SMLM) in deep tissue imaging, primarily due to depth-dependent aberrations caused by refractive index (RI) mismatch, present a significant challenge in achieving high-resolution images at greater depths. To extend the imaging depth, we optimized the imaging buffer of SMLM with the RI matched to that of the objective immersion medium and systematically evaluated five different RI-matched buffers, focusing on their impact on the blinking behavior of red-absorbing dyes and the quality of reconstructed super-resolution images. Particularly, we found that clear unobstructed brain imaging cocktails-based imaging buffer could match the RI of oil and was able to clear the tissue samples. With the help of the RI-matched imaging buffers, we showed high-quality dual-color 3D SMLM images with imaging depths ranging from a few micrometers to tens of micrometers in both cultured cells and sectioned tissue samples. This advancement offers a practical and accessible method for high-resolution imaging at greater depths without the need for specialized optical equipment or expertise.

ECLiPSE: a versatile classification technique for structural and morphological analysis of 2D and 3D single-molecule localization microscopy data

Single-molecule localization microscopy (SMLM) has gained widespread use for visualizing the morphology of subcellular organelles and structures with nanoscale spatial resolution. However, analysis tools for automatically quantifying and classifying SMLM images have lagged behind. Here we introduce Enhanced Classification of Localized Point clouds by Shape Extraction (ECLiPSE), an automated machine learning analysis pipeline specifically designed to classify cellular structures captured through two-dimensional or three-dimensional SMLM. ECLiPSE leverages a comprehensive set of shape descriptors, the majority of which are directly extracted from the localizations to minimize bias during the characterization of individual structures. ECLiPSE has been validated using both unsupervised and supervised classification on datasets, including various cellular structures, achieving near-perfect accuracy. We apply two-dimensional ECLiPSE to classify morphologically distinct protein aggregates relevant for neurodegenerative diseases. Additionally, we employ three-dimensional ECLiPSE to identify relevant biological differences between healthy and depolarized mitochondria. ECLiPSE will enhance the way we study cellular structures across various biological contexts.

Measurement precision bounds on aberrated single-molecule emission patterns.

Single-molecule localization microscopy (SMLM) has revolutionized the study of biological phenomena by providing exquisite nanoscale spatial resolution. However, optical aberrations induced by sample and system imperfections distort the single-molecule emission patterns (i.e. PSFs), leading to reduced precision and resolution of SMLM, particularly in three-dimensional (3D) applications. While various methods, both analytical and instrumental, have been employed to mitigate these aberrations, a comprehensive analysis of how different types of commonly encountered aberrations affect single-molecule experiments and their image formation remains missing. In this study, we addressed this gap by conducting a quantitative study of the theoretical precision limit for position and wavefront distortion measurements in the presence of aberrations. Leveraging Fisher information and Cramér-Rao lower bound (CRLB), we quantitively analyzed and compared the effects of different aberration types, including index mismatch aberrations, on localization precision in both biplane and astigmatism 3D modalities as well as 2D SMLM imaging. Furthermore, we studied the achievable wavefront estimation precision from aberrated single-molecule emission patterns, a pivot step for successful adaptive optics in SMLM through thick specimens. This analysis lays a quantitative foundation for the development and application of SMLM in whole-cells, tissues and with a large field of view, providing in-depth insights into the behavior of different aberration types in single-molecule imaging and thus generating theoretical guidelines for developing highly efficient aberration correction strategies and enhancing the precision and reliability of 3D SMLM.

Anisotropic DBSCAN for 3D SMLM Data Clustering.

Single-molecule localization microscopy (SMLM) advanced biological discoveries beyond the diffraction limit. Various implementations enable 3D SMLM to reconstruct volumetric cell images. Yet, the inherent anisotropic point spread function of optical microscopes often limits the localization precision in the axial direction compared to the lateral precision. Such localization anisotropy could also expand spherical cellular structures to ellipsoidal cellular structures. Structure identification, however, is often performed using DBSCAN cluster algorithms, considering an isotropic search volume. Here, we show that an anisotropic DBSCAN search volume identifies anisotropic clusters more reliably using simulated ground truth data sets. Given experimental localization precisions, we suggest optimized search parameters based on an expanded computational grid search and show an enhanced performance of anisotropic DBSCAN amidst variations in localization precision. We demonstrate the capability of anisotropic DBSCAN on experimental data and anticipate that the algorithm allows for a more rigorous identification of clusters in cells, considering the anisotropic localization precisions of astigmatism-based 3D SMLM.

Temporal-Focusing Multiphoton Excitation Single-Molecule Localization Microscopy Using Spontaneously Blinking Fluorophores.

Single-molecule localization microscopy (SMLM) based on temporal-focusing multiphoton excitation (TFMPE) and single-wavelength excitation is used to visualize the three-dimensional (3D) distribution of spontaneously blinking fluorophore-labeled subcellular structures in a thick specimen with a nanoscale-level spatial resolution. To eliminate the photobleaching effect of unlocalized molecules in out-of-focus regions for improving the utilization rate of the photon budget in 3D SMLM imaging, SMLM with single-wavelength TFMPE achieves wide-field and axially confined two-photon excitation (TPE) of spontaneously blinking fluorophores. TPE spectral measurement of blinking fluorophores is then conducted through TFMPE imaging at a tunable excitation wavelength, yielding the optimal TPE wavelength for increasing the number of detected photons from a single blinking event during SMLM. Subsequently, the TPE fluorescence of blinking fluorophores is recorded to obtain a two-dimensional TFMPE-SMLM image of the microtubules in cancer cells with a localization precision of 18±6 nm and an overall imaging resolution of approximately 51 nm, which is estimated based on the contribution of Nyquist resolution and localization precision. Combined with astigmatic imaging, the system is capable of 3D TFMPE-SMLM imaging of brain tissue section of a 5XFAD transgenic mouse with the pathological features of Alzheimer's disease, revealing the distribution of neurotoxic amyloid-beta peptide deposits.

Large-FOV 3D localization microscopy by spatially variant point spread function generation.

Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify the modeling of the imaging system. However, for large fields of view (FOV) imaging, it becomes important to account for the spatially variant nature of the PSF. Here, we propose an accurate and fast principal components analysis-based field-dependent 3D PSF generator (PPG3D) and localizer for LM. Through simulations and experimental three-dimensional (3D) single-molecule localization microscopy (SMLM), we demonstrate the effectiveness of PPG3D, enabling super-resolution imaging of mitochondria and microtubules with high fidelity over a large FOV. A comparison of PPG3D with a shift-variant PSF generator for 3D LM reveals a threefold improvement in accuracy. Moreover, PPG3D is approximately 100 times faster than existing PSF generators, when used in image plane-based interpolation mode. Given its user-friendliness, we believe that PPG3D holds great potential for widespread application in SMLM and other imaging modalities.

Quantifying nanoscopic alterations associated with mitochondrial dysfunction using three-dimensional single-molecule localization microscopy.

Mitochondrial morphology provides unique insights into their integrity and function. Among fluorescence microscopy techniques, 3D super-resolution microscopy uniquely enables the analysis of mitochondrial morphological features individually. However, there is a lack of tools to extract morphological parameters from super-resolution images of mitochondria. We report a quantitative method to extract mitochondrial morphological metrics, including volume, aspect ratio, and local protein density, from 3D single-molecule localization microscopy images, with single-mitochondrion sensitivity. We validated our approach using simulated ground-truth SMLM images of mitochondria. We further tested our morphological analysis on mitochondria that have been altered functionally and morphologically in controlled manners. This work sets the stage to quantitatively analyze mitochondrial morphological alterations associated with disease progression on an individual basis.

High-throughput single-molecule localization microscopy: Potential clinical applications.

High-throughput single-molecule localization microscopy: Potential clinical applications.

Purification Analysis, Intracellular Tracking, and Colocalization of Extracellular Vesicles Using Atomic Force and 3D Single-Molecule Localization Microscopy.

Extracellular vesicles (EVs) play a key role in cell-cell communication and thus have great potential to be utilized as therapeutic agents and diagnostic tools. In this study, we implemented single-molecule microscopy techniques as a toolbox for a comprehensive characterization as well as measurement of the cellular uptake of HEK293T cell-derived EVs (eGFP-labeled) in HeLa cells. A combination of fluorescence and atomic force microscopy revealed a fraction of 68% fluorescently labeled EVs with an average size of ∼45 nm. Two-color single-molecule fluorescence microscopy analysis elucidated the 3D dynamics of EVs entering HeLa cells. 3D colocalization analysis of two-color direct stochastic optical reconstruction microscopy (dSTORM) images revealed that 25% of EVs that experienced uptake colocalized with transferrin, which has been linked to early recycling of endosomes and clathrin-mediated endocytosis. The localization analysis was combined with stepwise photobleaching, providing a comparison of protein aggregation outside and inside the cells.

Deep learning driven adaptive optics for single molecule localization microscopy.

Single molecule localization microscopy (SMLM) forms super-resolution images with a resolution of several to tens of nanometers, by detecting isolated single molecule emission patterns and localizing the centers of individual probes. However, the inhomogeneous refractive indices distort and blur single molecule emission patterns, reduce the information content carried by each detected photon, increase localization uncertainty, and thus cause significant resolution loss, which is irreversible by post-processing. To compensate tissue induced aberrations, conventional sensorless adaptive optics methods rely on iterative mirror-changes and image-quality metrics to compensate aberrations. However, these metrics result in inconsistent, and sometimes opposite, metric responses which fundamentally limited the efficacy of these approaches for aberration correction in tissues. Bypassing the previous iterative trial-then-evaluate processes, we developed deep learning driven adaptive optics (DL-AO) for SMLM to allow direct inference of wavefront distortion and near real-time compensation. Our trained deep neural network (DNN) monitors the individual emission patterns from single molecule experiments, infers their shared wavefront distortion, feeds the estimates through a dynamic filter (Kalman), and drives a deformable mirror to compensate sample induced aberrations. The method simultaneously estimates and compensates 28 types of wavefront deformation shapes, restores single molecule emission patterns approaching the conditions untouched by specimen, and improves the resolution and fidelity of 3D SMLM through brain tissues over 130 µm, with as few as 3-20 mirror changes.

Single molecule imaging simulations with advanced fluorophore photophysics

Advanced fluorescence imaging techniques such as single-molecule localization microscopy (SMLM) fundamentally rely on the photophysical behavior of the employed fluorophores. This behavior is generally complex and impacts data quality in a subtle manner. A simulation software named Single-Molecule Imaging Simulator (SMIS) is introduced that simulates a widefield microscope and incorporates fluorophores with their spectral and photophysical properties. With SMIS, data collection schemes combining 3D, multicolor, single-particle-tracking or quantitative SMLM can be implemented. The influence of advanced fluorophore characteristics, imaging conditions, and environmental parameters can be evaluated, facilitating the design of real experiments and their proper interpretation.

SOLEIL: single-objective lens inclined light sheet localization microscopy.

High-NA light sheet illumination can improve the resolution of single-molecule localization microscopy (SMLM) by reducing the background fluorescence. These approaches currently require custom-made sample holders or additional specialized objectives, which makes the sample mounting or the optical system complex and therefore reduces the usability of these approaches. Here, we developed a single-objective lens-inclined light sheet microscope (SOLEIL) that is capable of 2D and 3D SMLM in thick samples. SOLEIL combines oblique illumination with point spread function PSF engineering to enable dSTORM imaging in a wide variety of samples. SOLEIL is compatible with standard sample holders and off-the-shelve optics and standard high NA objectives. To accomplish optimal optical sectioning we show that there is an ideal oblique angle and sheet thickness. Furthermore, to show what optical sectioning delivers for SMLM we benchmark SOLEIL against widefield and HILO microscopy with several biological samples. SOLEIL delivers in 15 μm thick Caco2-BBE cells a 374% higher intensity to background ratio and a 54% improvement in the estimated CRLB compared to widefield illumination, and a 184% higher intensity to background ratio and a 20% improvement in the estimated CRLB compared to HILO illumination.

Direct-laser writing for subnanometer focusing and single-molecule imaging

Two-photon direct laser writing is an additive fabrication process that utilizes two-photon absorption of tightly focused femtosecond laser pulses to implement spatially controlled polymerization of a liquid-phase photoresist. Two-photon direct laser writing is capable of nanofabricating arbitrary three-dimensional structures with nanometer accuracy. Here, we explore direct laser writing for high-resolution optical microscopy by fabricating unique 3D optical fiducials for single-molecule tracking and 3D single-molecule localization microscopy. By having control over the position and three-dimensional architecture of the fiducials, we improve axial discrimination and demonstrate isotropic subnanometer 3D focusing (<0.8 nm) over tens of micrometers using a standard inverted microscope. We perform 3D single-molecule acquisitions over cellular volumes, unsupervised data acquisition and live-cell single-particle tracking with nanometer accuracy.

Direct supercritical angle localization microscopy for nanometer 3D superresolution

3D single molecule localization microscopy (SMLM) has emerged as a powerful method for structural cell biology, as it allows probing precise positions of proteins in cellular structures at a resolution of tens of nanometers in both 2D and 3D. Popular approaches to 3D imaging include PSF engineering to encode the position of single-emitters in its shape. However, such methods lead to worse localization precision axially than laterally. Complex localization microscopy techniques, such as iPALM, can provide isotropic resolution at the cost of a complex instrument.

Numerical simulation study of three-dimensional high-density single molecule localization microscopy based on orthogonal astigmatism

Single molecule localization microscopy (SMLM) detects and locates sparsely luminous single fluorescent molecules to achieve super-resolution imaging at nanoscale spatial resolution. In order to improve the temporal resolution, it is necessary to increase the density of the simultaneously emitting molecules. However, with the increase of the density, the point spread function (PSF) of different molecules will overlap severely on the detector, resulting in reduced spatial resolution, especially for three-dimensional (3D) SMLM. To solve this problem, a high density 3D-SMLM imaging method based on orthogonal astigmatism is proposed. Analysis and numerical simulation study for the method are carried out and presented. The main idea of the proposed orthogonal astigmatic method is to split the collected fluorescence in a SMLM microscope into two beams, each of which passes through a separate channel with a cylindrical lens and arrives at a specific region on the same detector. The two cylindrical lenses have the same optical parameters, but their orientations are set to be orthogonal to each other. They are used to obtain both positive and negative astigmatic PSF images of the same fluorescent molecule. Then, a linear projection model of the imaging process is established, and the 3D localization of the fluorescent molecules is realized by using a compression sensing algorithm. The results show that the two orthogonal cylindrical lenses produce a pair of astigmatic PSFs for one single molecule so that different PSF pairs between different molecules have lower mutual correlation, and thus the 3D localization accuracy for high density imaging can be significantly improved as compared with traditional astigmatic method, in which one single cylindrical lens is used. The larger the defocusing degree, the greater the shape difference between the two astigmatic PSFs is, and the more obvious this advantage.

Studying the Dynamics of Chromatin-Binding Proteins in Mammalian Cells Using Single-Molecule Localization Microscopy.

Single-molecule localization microscopy (SMLM) allows the super-resolved imaging of proteins within mammalian nuclei at spatial resolutions comparable to that of a nucleosome itself (~20nm). The technique is therefore well suited to the study of chromatin structure. Fixed-cell SMLM has already allowed temporal "snapshots" of how proteins are arranged on chromatin within mammalian nuclei. In this chapter, we focus on how recent developments, for example in selective plane illumination, 3D SMLM, and protein labeling, have led to a range of live-cell SMLM studies. We describe how to carry out single-particle tracking (SPT) of single proteins and, by analyzing their diffusion parameters, how to determine whether proteins interact with chromatin, diffuse freely, or do both. We can study the numbers of proteins that interact with chromatin and also determine their residence time on chromatin. We can determine whether these proteins form functional clusters within the nucleus as well as whether they form specific nuclear structures.

3D Structure From 2D Microscopy Images Using Deep Learning.

Understanding the structure of a protein complex is crucial in determining its function. However, retrieving accurate 3D structures from microscopy images is highly challenging, particularly as many imaging modalities are two-dimensional. Recent advances in Artificial Intelligence have been applied to this problem, primarily using voxel based approaches to analyse sets of electron microscopy images. Here we present a deep learning solution for reconstructing the protein complexes from a number of 2D single molecule localization microscopy images, with the solution being completely unconstrained. Our convolutional neural network coupled with a differentiable renderer predicts pose and derives a single structure. After training, the network is discarded, with the output of this method being a structural model which fits the data-set. We demonstrate the performance of our system on two protein complexes: CEP152 (which comprises part of the proximal toroid of the centriole) and centrioles.

Fast 3D localization algorithm for high-density molecules based on multiple measurement vector compressed sensing

Single-molecule localization microscopy (SMLM) achieves super-resolution based on a series of images of sparsely distributed molecules, which makes it time-consuming. To improve the temporal resolution, several high-density localization algorithms have been developed, and the computational speed is considered one of the important factors for evaluating their performance. In this study, for the first time, a three-dimensional (3D) multiple measurement vector compressed sensing (3D-MMV-CS) model was constructed and then solved using a multiple sparse Bayesian learning algorithm to realize 3D single-molecule localization and obtain 3D super-resolved fluorescence images. The 3D-MMV-CS successfully recovered a 3D super-resolved image from a series of simulated high-density fluorescence images. The computation time for reconstruction only increases marginally with an increase in the number of raw images and approximately stays constant with an increase in the density of fluorescent molecules. Simulation results demonstrated that, when the number of raw images reaches 1900, compared with the other two algorithms L1-Homotopy and compressed sensing STORM (CSSTORM) with CVX, the 3D-MMV-CS method is 100 times faster than the former and five orders of magnitude faster than the latter. This outstanding performance in computation speed would allow the 3D-MMV-CS approach to be routinely applicable to SMLM. Validation test of the 3D-MMV-CS method was performed successfully with experimental data from an astigmatism-based 3D SMLM system.

Real-time 3D single-molecule localization microscopy analysis using lookup tables.

Herein, we present a new algorithm for real-time analysis of 3D single molecule localization microscopy images with a small impact on fitting accuracy using lookup-tables with discrete xyz-positions. The algorithm realizes real-time visualization during acquisition. We demonstrate its performance on simulated and measured data. Additionally, combining real-time fitting with a feedback loop controlling the activation laser pulse keeps the number of emitters per image frame constant.