Experimental Point Spread Function Research Articles

Three-dimensional thermophotonic super-resolution imaging by spatiotemporal diffusion reversal method.

Spatial super-resolution in thermophotonic imaging was achieved using a combination of spatial second-derivative forming, spatial gradient adaptive filtering, and Richardson-Lucy deconvolution in conjunction with the construction of an experimental point spread function. When implemented through enhanced truncation-correlation photothermal coherence tomography (eTC-PCT), it was possible to restore blurred infrared thermophotonic images to their prediffusion optical resolution state. This modality was tested in various biological applications and proved to be capable of imaging fine axial cracks in human teeth, well-patterned anatomical subsurface structures of a mouse brain, and neovascularization in a mouse thigh due to the rapid proliferation of cancer cells. This modality was found to be immune to optical scattering and could reveal the true spatial extent of biological features at subsurface depths that conventional thermal imaging cannot reach because of limitations imposed by the physics of spreading diffusion.

Enhancing obSTORM imaging performance with cubic spline PSF modeling.

Oblique plane microscopy-based single molecule localization microscopy (obSTORM) has shown great potential for super-resolution imaging of thick biological specimens. Despite its compatibility with tissues and small animals, prior uses of the Gaussian point spread function (PSF) model have resulted in limited imaging resolution and a narrow axial localization range. This is due to the poor fit of the Gaussian PSF model with the actual PSF shapes in obSTORM. To overcome these limitations, we have employed cubic splines for a more accurate modeling of the experimental PSF shapes. This refined PSF model enhances three-dimensional localization precision, leading to significant improvements in obSTORM imaging of mouse retina tissues, such as an approximately 1.2 times increase in imaging resolution, seamless stitching of single molecules between adjacent optical sections, and a doubling of the sectional interval in volumetric obSTORM imaging due to the extended axial range of usable section thickness. The cubic spline PSF model thus offers a path towards more accurate and faster volumetric obSTORM imaging of biological specimens.

Miniature color camera via flat hybrid meta-optics.

The race for miniature color cameras using flat meta-optics has rapidly developed the end-to-end design framework using neural networks. Although a large body of work has shown the potential of this methodology, the reported performance is still limited due to fundamental limitations coming from meta-optics, mismatch between simulated and resultant experimental point spread functions, and calibration errors. Here, we use a HIL optics design methodology to solve these limitations and demonstrate a miniature color camera via flat hybrid meta-optics (refractive + meta-mask). The resulting camera achieves high-quality full-color imaging for a 5-mm aperture optics with a focal length of 5 mm. We observed a superior quality of the images captured by the hybrid meta-optical camera compared to a compound multi-lens optics of a mirrorless commercial camera.

SMITE (single molecule imaging toolbox extraordinaire), a MATLAB package.

We present SMITE, a MATLAB package that features analysis tools for fluorescence single molecule imaging with an emphasis on single molecule localization microscopy (SMLM) and single-particle tracking (SPT). Some C and CUDA code is embedded in the MATLAB infrastructure to provide CPU/GPU speed-ups for particularly expensive computations. SMITE is organized around two data structures: SMF (Single Molecule Fitting), containing parameters that completely define the data analysis, and SMD (Single Molecule Data), which contains the results of the analysis. The code is divided into namespaces and subclasses that group related concepts and functionality together. SMITE is designed so that low-level tools as well as higher-level application specific interfaces are easy to use, making use of the SMF and SMD data structures whenever possible. Interaction with these tools is via GUIs or scripting. Extensive internal documentation of the interfaces and many usage examples are provided to aid the user. Code coverage includes mature SMLM data analysis techniques (applying gain and offset corrections to raw data, finding localizations, thresholding localizations based on various criteria, frame connection and drift correction), SMLM/SPT simulations, sophisticated SPT analyses, post-processing clustering and statistical analyses (e.g., Bayesian grouping of localizations for removing repeat localizations from the same emitter, diffusion analysis, hidden Markov models for characterizing dimers in SPT results), a variety of visualizations, experimental point spread function creation and characterization. SMITE is available via GitHub: https://github.com/LidkeLab/smite

Development of a 3D-immunofluorescence analysis for sensory nerve endings in human ligaments

BackgroundThe analysis of ligamentous mechanoreceptors is difficult due to a high amount of unclassifiable mechanoreceptors, which result from incomplete visualization through limited microscopic techniques. New methodThe method was developed using dorsal intercarpal ligaments and dorsal regions of the scapholunate interosseous ligament from human cadaver wrists. Consecutive 70 µm thick cryosections were stained with immunofluorescence markers for protein S100B, neurotrophin receptor p75 (p75), protein gene product 9.5 (PGP 9.5) and 4′,6-diamidino-2-phenylindole (DAPI). 3D images of sensory nerve endings were obtained using a confocal laser scanning microscope. Experimental point spread functions (PSF) were used to deconvolve images. Sensory nerve endings were localised in each section plane and classified according to Freeman and Wyke. Finally, confocal data was visualized as 3D-images. ResultsThe method produced excellent image quality, revealing detailed three-dimensional structures. The created 3D-model of sensory nerve endings could be analyzed in all three dimensions, augmenting visualization of the form and immunoreactive pattern of sensory nerve endings. Deconvolution with experimentally measured PSFs aided in enhancing image quality. Comparison with existing methodsUsing a triple immunofluorescent staining method allows to visualize the structure of sensory nerve endings more precisely than techniques with serial analysis of different monostaining of neural markers. Imaging in three dimensions enhances morphologic details, which are limited in 2D-microscopy. Conclusion3D-triple immunofluorescence produces high quality visualization of mechanoreceptors, thereby improving their analysis. As an elaborate technique, it is ideal for defined research questions concerning the microstructure of sensory nerve endings.

Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging.

End-to-end optimization of diffractive optical elements (DOEs) profile through a digital differentiable model combined with computational imaging have gained an increasing attention in emerging applications due to the compactness of resultant physical setups. Despite recent works have shown the potential of this methodology to design optics, its performance in physical setups is still limited and affected by manufacturing artefacts of DOE, mismatch between simulated and resultant experimental point spread functions, and calibration errors. Additionally, the computational burden of the digital differentiable model to effectively design the DOE is increasing, thus limiting the size of the DOE that can be designed. To overcome the above mentioned limitations, a co-design of hybrid optics and image reconstruction algorithm is produced following the end-to-end hardware-in-the-loop strategy, using for optimization a convolutional neural network equipped with quantitative and qualitative loss functions. The optics of the imaging system consists on the phase-only spatial light modulator (SLM) as DOE and refractive lens. SLM phase-pattern is optimized by applying the Hardware-in-the-loop technique, which helps to eliminate the mismatch between numerical modelling and physical reality of image formation as light propagation is not numerically modelled but is physically done. Comparison with compound multi-lens optics of a last generation smartphone and a mirrorless commercial cameras show that the proposed system is advanced in all-in-focus sharp imaging for a depth range 0.4-1.9 m.

Blind Image Restoration Enhances Digital Autoradiographic Imaging of Radiopharmaceutical Tissue Distribution.

Digital autoradiography (DAR) is a powerful tool to quantitatively determine the distribution of a radiopharmaceutical within a tissue section and is widely used in drug discovery and development. However, the low image resolution and significant background noise can result in poor correlation, even errors, between radiotracer distribution, anatomic structure, and molecular expression profiles. Differing from conventional optical systems, the point-spread function in DAR is determined by properties of radioisotope decay, phosphor, and digitizer. Calibration of an experimental point-spread function a priori is difficult, prone to error, and impractical. We have developed a content-adaptive restoration algorithm to address these problems. Methods: We model the DAR imaging process using a mixed Poisson-gaussian model and blindly restore the image by a penalized maximum-likelihood expectation-maximization algorithm (PG-PEM). PG-PEM implements a patch-based estimation algorithm with density-based spatial clustering of applications with noise to estimate noise parameters and uses L2 and Hessian Frebonius norms as regularization functions to improve performance. Results: First, PG-PEM outperformed other restoration algorithms at the denoising task (P < 0.01). Next, we implemented PG-PEM on preclinical DAR images (18F-FDG, treated mouse tumor and heart; 18F-NaF, treated mouse femur) and clinical DAR images (bone biopsy sections from 223RaCl2-treated castration-resistant prostate cancer patients). DAR images restored by PG-PEM of all samples achieved a significantly higher effective resolution and contrast-to-noise ratio and a lower SD of background (P < 0.0001). Additionally, by comparing the registration results between the clinical DAR images and the segmented bone masks from the corresponding histologic images, we found that the radiopharmaceutical distribution was significantly improved (P < 0.0001). Conclusion: PG-PEM is able to increase resolution and contrast while robustly accounting for DAR noise and demonstrates the capacity to be widely implemented to improve preclinical and clinical DAR imaging of radiopharmaceutical distribution.

Enhancing Direct Exoplanet Spectroscopy with Apodizing and Beam Shaping Optics

Direct exoplanet spectroscopy aims to measure the spectrum of an exoplanet while simultaneously minimizing the light collected from its host star. Isolating the planet light from the starlight improves the signal-to-noise ratio (S/N) per spectral channel when noise due to the star dominates, which may enable new studies of the exoplanet atmosphere with unprecedented detail at high spectral resolution (>30,000). However, the optimal instrument design depends on the flux level from the planet and star compared to the noise due to other sources, such as detector noise and thermal background. Here we present the design, fabrication, and laboratory demonstration of specially-designed optics to improve the S/N in two potential regimes in direct exoplanet spectroscopy with adaptive optics instruments. The first is a pair of beam-shaping lenses that increase the planet signal by improving the coupling efficiency into a single-mode fiber at the known position of the planet. The second is a grayscale apodizer that reduces the diffracted starlight for planets at small angular separations from their host star. The former especially increases S/N when dominated by detector noise or thermal background, while the latter helps reduce stellar noise. We show good agreement between the theoretical and experimental point spread functions in each case and predict the exposure time reduction (∼33%) that each set of optics provides in simulated observations of 51 Eridani b using the Keck Planet Imager and Characterizer instrument at W. M. Keck Observatory.

Accurate 4Pi single-molecule localization using an experimental PSF model.

Interferometric single-molecule localization microscopy (iPALM, 4Pi-SMS) uses multiphase interferometry to localize single fluorophores and achieves nanometer isotropic resolution in 3D. The current data analysis workflow, however, fails to reach the theoretical resolution limit due to the suboptimal localization algorithm. Here, we develop a method to fit an experimentally derived point spread function (PSF) model to the interference 4Pi-PSF. As the interference phase is not fixed with respect to the shape of the PSF, we decoupled the phase term in the model from the 3D position of the PSF. The fitter can reliably infer the interference period even without introducing astigmatism, reducing the complexity of the microscope. Using a spline-interpolated experimental PSF model and by fitting all phase images globally, we show on simulated data that we can achieve the theoretical limit of 3D resolution, the Cramér-Rao lower bound (CRLB), also for the 4Pi microscope.

Two simple criteria to estimate an objective’s performance when imaging in non design tissue clearing solutions

Tissue clearing techniques are undergoing a renaissance motivated by the need to image fluorescent neurons, and other cells, deep in the sample without physical sectioning. Optical transparency is achieved by equilibrating tissues with high refractive index (RI) solutions. When the microscope objective is not perfectly matched to the RI of the cleared sample, aberrations are introduced. We present two simple-to-calculate numerical criteria predicting: (i) the degradation in image quality (brightness and resolution) from optimal conditions of any clearing solution/objective combination; (ii) which objective, among several available, achieves the highest resolution in a given medium. We derived closed form approximations for image quality degradation versus RI mismatch and other parameters available to the microscopist, validated them with computed and measured aberrated point spread functions and by imaging fluorescent neurons in high RI solution. These approximations apply to the widefield fluorescent microscope but are also relevant to more advanced microscopes. Currently, to accurately predict the impact of RI mismatch-induced aberrations on imaging, the life scientist must examine theoretical or experimental point spread functions (PSFs) obtained under the optical configuration of interest. These criteria can be used to select a suitable objective for the chosen clearing method (particularly when subject to budget constraints) or to tweak a clearing solution RI to the available objectives. Even with a nominally optimal objective, one may wish to assess the impact of any small unavoidable mismatches.

Information-rich localization microscopy through machine learning

Recent years have witnessed the development of single-molecule localization microscopy as a generic tool for sampling diverse biologically relevant information at the super-resolution level. While current approaches often rely on the target-specific alteration of the point spread function to encode the multidimensional contents of single fluorophores, the details of the point spread function in an unmodified microscope already contain rich information. Here we introduce a data-driven approach in which artificial neural networks are trained to make a direct link between an experimental point spread function image and its underlying, multidimensional parameters, and compare results with alternative approaches based on maximum likelihood estimation. To demonstrate this concept in real systems, we decipher in fixed cells both the colors and the axial positions of single molecules in regular localization microscopy data.

Incoherent holographic camera based on Michelson Interferometer

Incoherent holography is a hybrid imaging technology with coherent and incoherent processes. Compared with coherent holography, incoherent holography can alleviate coherent noise and speckle noise, and thus achieve higher signal-to-noise ratio. In this article, the incoherent holographic camera based on Michelson interferometer is built, which consists of a telescope, a beam-splitting cube, two plane mirrors, two lenses and a monochromatic charge coupled device. The point spread function (PSF) of the system is determined for the first time according to the Fresnel diffraction theory. Both the simulated and experimental 2D PSFs are coincident with each other. The three-step generalized phase shift interferometry instead of the piezo-actuator is used to eliminate zero-order and twin images. The two one-jiao coins with 15 mm apart and a toy house are recorded under xenon lamp illumination. The reconstructed image clearly demonstrates that the system has the ability to obtain spatial 3D information of an object.

Real-time 3D single-molecule localization using experimental point spread functions.

We present a real-time fitter for 3D single-molecule localization microscopy using experimental point spread functions (PSFs) that achieves optimal 3D resolution on any microscope and is compatible with any PSF engineering approach. This allowed us to image cellular structures with a 3D resolution unprecedented for astigmatic PSFs. The fitter compensates for most optical aberrations and makes accurate 3D superresolution microscopy broadly accessible, even on standard microscopes without dedicated 3D optics.

MapX: 2D XRF for Planetary Exploration - Image Formation and Optic Characterization

Map-X is a planetary instrument concept for 2D X-Ray Fluorescence (XRF) spectroscopy. The instrument is placed directly on the surface of an object and held in a fixed position during the measurement. The formation of XRF images on the CCD detector relies on a multichannel optic configured for 1:1 imaging and can be analyzed through the point spread function (PSF) of the optic. The PSF can be directly measured using a micron-sized monochromatic X-ray source in place of the sample. Such PSF measurements were carried out at the Stanford Synchrotron and are compared with ray tracing simulations. It is shown that artifacts are introduced by the periodicity of the PSF at the channel scale and the proximity of the CCD pixel size and the optic channel size. A strategy of sub-channel random moves was used to cancel out these artifacts and provide a clean experimental PSF directly usable for XRF image deconvolution.

Multiline 3D beamforming using micro-beamformed datasets for pediatric transesophageal echocardiography

Until now, no matrix transducer has been realized for 3D transesophageal echocardiography (TEE) in pediatric patients. In 3D TEE with a matrix transducer, the biggest challenges are to connect a large number of elements to a standard ultrasound system, and to achieve a high volume rate (>200 Hz). To address these issues, we have recently developed a prototype miniaturized matrix transducer for pediatric patients with micro-beamforming and a small central transmitter. In this paper we propose two multiline parallel 3D beamforming techniques (µBF25 and µBF169) using the micro-beamformed datasets from 25 and 169 transmit events to achieve volume rates of 300 Hz and 44 Hz, respectively. Both the realizations use angle-weighted combination of the neighboring overlapping sub-volumes to avoid artifacts due to sharp intensity changes introduced by parallel beamforming. In simulation, the image quality in terms of the width of the point spread function (PSF), lateral shift invariance and mean clutter level for volumes produced by µBF25 and µBF169 are similar to the idealized beamforming using a conventional single-line acquisition with a fully-sampled matrix transducer (FS4k, 4225 transmit events). For completeness, we also investigated a 9 transmit-scheme (3 × 3) that allows even higher frame rates but found worse B-mode image quality with our probe. The simulations were experimentally verified by acquiring the µBF datasets from the prototype using a Verasonics V1 research ultrasound system. For both µBF169 and µBF25, the experimental PSFs were similar to the simulated PSFs, but in the experimental PSFs, the clutter level was ~10 dB higher. Results indicate that the proposed multiline 3D beamforming techniques with the prototype matrix transducer are promising candidates for real-time pediatric 3D TEE.

High precision wavefront control in point spread function engineering for single emitter localization.

Point spread function (PSF) engineering is used in single emitter localization to measure the emitter position in 3D and possibly other parameters such as the emission color or dipole orientation as well. Advanced PSF models such as spline fits to experimental PSFs or the vectorial PSF model can be used in the corresponding localization algorithms in order to model the intricate spot shape and deformations correctly. The complexity of the optical architecture and fit model makes PSF engineering approaches particularly sensitive to optical aberrations. Here, we present a calibration and alignment protocol for fluorescence microscopes equipped with a spatial light modulator (SLM) with the goal of establishing a wavefront error well below the diffraction limit for optimum application of complex engineered PSFs. We achieve high-precision wavefront control, to a level below 20 mλ wavefront aberration over a 30 minute time window after the calibration procedure, using a separate light path for calibrating the pixel-to-pixel variations of the SLM, and alignment of the SLM with respect to the optical axis and Fourier plane within 3 μm (x/y) and 100 μm (z) error. Aberrations are retrieved from a fit of the vectorial PSF model to a bead z-stack and compensated with a residual wavefront error comparable to the error of the SLM calibration step. This well-calibrated and corrected setup makes it possible to create complex '3D+λ' PSFs that fit very well to the vectorial PSF model. Proof-of-principle bead experiments show precisions below 10 nm in x, y, and λ, and below 20 nm in z over an axial range of 1 μm with 2000 signal photons and 12 background photons.

Edge contrast enhancement of Fresnel incoherent correlation holography (FINCH) microscopy by spatial light modulator aided spiral phase modulation

Digital holographic microscopy (DHM) is one of the most effective methods in imaging the weakly-scattering objects, such as small colloidal particles and most biological cells. Compared to phase contrast and differential interference contrast microscopy, DHM cannot only visualize but quantify these phase objects. In this work, a spiral phase modulated FINCH microscope was implemented. The core of the system is an in-line incoherent interferometer composed of a spatial light modulator (SLM) and a charge-coupled device. In order to enhance image contrast, the SLM was space-division multiplexed by a helical lens and a conventional lens. To study the properties of this vortex imaging system, the precise mathematical model of the Point Spread Function (PSF), which describes the intensity distribution in digital image of the system’s response to a point source, is determined for the first time from the view of wave optics. The experimental 2D PSF agrees well with that of simulated one. When the system is used for biological microscopic imaging the enhancement of edge contrast and the enlargement of field of view are obtained without loss of resolution.

Spatial-domain filter enhanced subtraction microscopy and application to mid-IR imaging.

We have experimentally investigated the enhancement in spatial resolution by image subtraction in mid-infrared central solid-immersion lens (c-SIL) microscopy. The subtraction exploits a first image measured with the c-SIL point-spread function (PSF) realized with a Gaussian beam and a second image measured with the beam optically patterned by a silicon π-step phase plate, to realize a centrally hollow PSF. The intense sides lobes in both PSFs that are intrinsic to the SIL make the conventional weighted subtraction methods inadequate. A spatial-domain filter with a kernel optimized to match both experimental PSFs in their periphery was thus developed to modify the first image prior to subtraction, and this resulted in greatly improved performance, with polystyrene beads 1.4 ± 0.1 µm apart optically resolved with a mid-IR wavelength of 3.4 µm in water. Spatial-domain filtering is applicable to other PSF pairs, and simulations show that it also outperforms conventional subtraction methods for the Gaussian and doughnut beams widely used in visible and near-IR microscopy.

Experimental validation of a customized phase mask designed to enable efficient computational optical sectioning microscopy through wavefront encoding.

In this paper, wavefront-encoded (WFE) computational optical sectioning microscopy (COSM) using a fabricated square cubic (SQUBIC) phase mask, designed to render the system less sensitive to depth-induced aberration, is investigated. The WFE-COSM system is characterized by a point spread function (PSF) that does not vary as rapidly with imaging depth compared to the conventional system. Thus, in WFE-COSM, image restoration from large volumes can be achieved using computationally efficient space-invariant (SI) algorithms, thereby avoiding the use of depth-variant algorithms. The fabricated SQUBIC phase mask was first evaluated and found to have a 75% fidelity compared to the theoretical design; it was then integrated in a commercial wide-field microscope to implement a WFE-COSM system. Evaluation of the WFE-COSM system is demonstrated with comparative studies of theoretical and experimental PSFs and simulated and measured images of spherical shells located at different depths in a test sample. These comparisons show that PSF and imaging models capture major trends in experimental data with a 99% correlation between forward image intensity distribution in experimental and simulated images of spherical shells. Our experimental SI restoration results demonstrate that the WFE-COSM system achieves more than a twofold performance improvement over the conventional system of up to a 65 μm depth below the coverslip investigated in this study.

MreB Senses Local Gaussian Curvature to Pattern Rod-Like Growth of the Bacterial Cell Wall

Over the past few years we have developed an image-processing framework that allows us to extract precise 3D shapes of bacterial cells from fluorescence microscopy data. This active mesh approach minimizes the difference between an observed Z-stack and model shapes that have been convolved with the experimental point spread function. From these xyz coordinates, we calculate geometric parameters such as local curvatures, surface areas, and the relative enrichment of fluorescent signals. Our fluorescent signal is the concentration of the bacterial actin homolog MreB. We have integrated a fluorescent sandwich fusion of MreB with msfGFP at the native locus. This construct has unperturbed mass-doubling times and a proper rod-like shape.We have previously shown in both two and three dimensions that MreB is enriched at negative curvatures and away from positive curvatures [Ursell et al. PNAS 2014]. Here we show in the straight rod Escherichia coli and the curved rod Caulobacter crescentus that MreB senses local surface curvature and, as a result, helps ensure rod-like growth of the cell. The curvature sensed by Escherchia coli MreB is the Gaussian curvature of the surface, the product of the two principal curvatures. Using MreB point mutants that have altered curvature sensitivities, we are testing the hypothesis that cells straighten deformations by patterning growth at the proper geometry. To stably generate their comma shaped geometry, Caulobacter crescentus have developed ways to overcome this straightening mechanism. The enrichment profile of CcMreB shows a plateau in enrichment at small, positive Gaussian curvature values that depends on the protein crescentin. In addition to its structural role in bending the cell, we propose a mechanism whereby crescentin alters the curvature preference of MreB and allows the stabile propagation of the Caulobacter shape.