3D Point Spread Function Research Articles

Augmented 3D super-resolution of fluorescence-free nanoparticles using enhanced dark-field illumination based on wavelength-modulation and a least-cubic algorithm.

Augmented three-dimensional (3D) subdiffraction-limited resolution of fluorescence-free single-nanoparticles was achieved with wavelength-dependent enhanced dark-field (EDF) illumination and a least-cubic algorithm. Various plasmonic nanoparticles on a glass slide (i.e., gold nanoparticles, GNPs; silver nanoparticles, SNPs; and gold nanorods, GNRs) were imaged and sliced in the z-direction to a thickness of 10 nm. Single-particle images were then compared with simulation data. The 3D coordinates of individual GNP, SNP, and GNR nanoparticles (x, y, z) were resolved by fitting the data with 3D point spread functions using a least-cubic algorithm and collation. Final, 3D super-resolution microscopy (SRM) images were obtained by resolving 3D coordinates and their Cramér-Rao lower bound-based localization precisions in an image space (530 nm × 530 nm × 300 nm) with a specific voxel size (2.5 nm × 2.5 nm × 5 nm). Compared with the commonly used least-square method, the least-cubic method was more useful for finding the center in asymmetric cases (i.e., nanorods) with high precision and accuracy. This novel 3D fluorescence-free SRM technique was successfully applied to resolve the positions of various nanoparticles on glass and gold nanospots (in vitro) as well as in a living single cell (in vivo) with subdiffraction limited resolution in 3D.

2D Sub-pixel Point Spread Function Measurement Using a Virtual Point-Like Source

2D point spread function (PSF) is a commonly used measure to assess the quality of various imaging systems. The most convenient way of 2D PSF measurement is taking an image of a light source with its size well below the diffraction limit of the imaging system. In this paper, we present a novel method that allows formation of such a virtual point-like source by a simple setup with a convex spherical mirror and a collimated light source. Sub-pixel 2D PSF measurements are possible by displacing the setup in sub-pixel steps. Comparison of the 1D modulation transfer functions estimated by the proposed method and the International Organization for Standardization (ISO) 12233 standard shows that the proposed method presents a viable alternative to the ISO 12233 standard. Furthermore, future work on calibration patterns and algorithms for sub-pixel 2D PSF estimation from a single image could benefit from the presented method, which provides ground truth sub-pixel 2D PSF measurements for real imaging systems.

Generalized recovery algorithm for 3D super-resolution microscopy using rotating point spread functions.

Super-resolution microscopy with phase masks is a promising technique for 3D imaging and tracking. Due to the complexity of the resultant point spread functions, generalized recovery algorithms are still missing. We introduce a 3D super-resolution recovery algorithm that works for a variety of phase masks generating 3D point spread functions. A fast deconvolution process generates initial guesses, which are further refined by least squares fitting. Overfitting is suppressed using a machine learning determined threshold. Preliminary results on experimental data show that our algorithm can be used to super-localize 3D adsorption events within a porous polymer film and is useful for evaluating potential phase masks. Finally, we demonstrate that parallel computation on graphics processing units can reduce the processing time required for 3D recovery. Simulations reveal that, through desktop parallelization, the ultimate limit of real-time processing is possible. Our program is the first open source recovery program for generalized 3D recovery using rotating point spread functions.

Can CT image deblurring improve finite element predictions at the proximal femur?

The aim of this study was to determine if a CT image deblurring algorithm can improve CT-based FE modelling accuracy at the proximal femur.Experimental data (CT scans of fourteen proximal fresh-frozen cadaveric femurs, non-destructive surface strain measurements in stance and sideways fall loading configurations on all femurs, and failure loads obtained in stance for seven specimens, in sideways fall for the other seven) were taken from a recent study (Schileo et al., 2014). An estimate of the 3D Point Spread Function for each CT scan was used within a deconvolution solver to perform the deblurring. The most proximal regions of three specimens were scanned using an HRpQCT scanner and compared to the original and deblurred CT images to quantify errors in bone contour estimates and determine correlation of intensity values within the bone contours. Subject-specific FE models of the proximal femur were generated. The accuracy of deblurred FE predictions against experimental measurements was compared to the published (non-deblurred) FE results.When compared to HRpQCT, CT deblurring led to lower mean surface distances (0.31 vs. 0.49mm) and higher CT intensity correlations with respect to the original CT. All indicators of strain prediction accuracy were significantly improved in deblurred FE models, more markedly at the femoral neck (peak error reduced by 38%). Failure load prediction, based on a simple elastic limit model, was also improved in deblurred FE models, although differently for stance and sideways fall loading conditions. In stance, correlation was unchanged, but specimen-wise errors were reduced (mean error 10% vs. 15%). In sideways fall, correlation notably increased (R2=0.95 vs. 0.81), despite a general overestimation of failure load.In summary, the proposed CT deblurring technique yielded moderate but significant improvements in FE predictions, and may thus be considered a first step toward the improvement of CT-based FE models of the human femur.

Restoration of Thickness, Density, and Volume for Highly Blurred Thin Cortical Bones in Clinical CT Images.

In clinical CT images containing thin osseous structures, accurate definition of the geometry and density is limited by the scanner's resolution and radiation dose. This study presents and validates a practical methodology for restoring information about thin bone structure by volumetric deblurring of images. The methodology involves 2 steps: a phantom-free, post-reconstruction estimation of the 3D point spread function (PSF) from CT data sets, followed by iterative deconvolution using the PSF estimate. Performance of 5 iterative deconvolution algorithms, blind, Richardson-Lucy (standard, plus Total Variation versions), modified residual norm steepest descent (MRNSD), and Conjugate Gradient Least-Squares were evaluated using CT scans of synthetic cortical bone phantoms. The MRNSD algorithm resulted in the highest relative deblurring performance as assessed by a cortical bone thickness error (0.18mm) and intensity error (150HU), and was subsequently applied on a CT image of a cadaveric skull. Performance was compared against micro-CT images of the excised thin cortical bone samples from the skull (average thickness 1.08±0.77mm). Error in quantitative measurements made from the deblurred images was reduced 82% (p<0.01) for cortical thickness and 55% (p<0.01) for bone mineral mass. These results demonstrate a significant restoration of geometrical and radiological density information derived for thin osseous features.

3D Single-Molecule Super-Resolution Fluorescence Microscopy with the Corkscrew Point Spread Function

The Moerner Laboratory has been engineering the point spread function (PSF) of a wide-field fluorescence microscope in order to obtain simultaneous X, Y, and Z spatial information from single molecules over a large Z-range. The new PSFs (e.g., the double-helix, tetrapod, and corkscrew PSFs) change their shapes or positions as the single-molecule emitter moves along the optical (Z) axis—thus conveying rich information about the Z position of the molecule. These approaches are relevant both for tracking individual biomolecules at low concentration as well as reconstructing super-resolution images by single-molecule localization microscopy. The corkscrew PSF is special because it behaves like a single Gaussian spot of light that moves along a U-shaped path as the emitter moves along the optical axis. Hence, the corkscrew PSF occupies less space on the camera as compared to our other 3D PSFs and allows us to image a sample more quickly at a higher density of localizations per camera frame.In this work, we use the corkscrew PSF to image a rectilinear array of nanoholes filled with fluorescent molecules to calibrate the three-dimensional response of our microscope. This involves mapping the 3D object space of molecules within a sample to the 2D image space on our camera. By sampling the microscope's response throughout the field of view, we account for field-dependent aberrations that would otherwise cause errors in the position measurements of single molecules. Recent progress in imaging biomolecules with the corkscrew PSF will be summarized.

Simple method based on intensity measurements for characterization of aberrations from micro-optical components.

We report a simple method, based on intensity measurements, for the characterization of the wavefront and aberrations produced by micro-optical focusing elements. This method employs the setup presented earlier in [Opt. Express 22, 13202 (2014)] for measurements of the 3D point spread function, on which a basic phase-retrieval algorithm is applied. This combination allows for retrieval of the wavefront generated by the micro-optical element and, in addition, quantification of the optical aberrations through the wavefront decomposition with Zernike polynomials. The optical setup requires only an in-motion imaging system. The technique, adapted for the optimization of micro-optical component fabrication, is demonstrated by characterizing a planoconvex microlens.

3D localization of high particle density images using sparse recovery.

If particles are too close in space, their images may be overlapped when they are observed with microscopes because of diffraction limitation, which makes them difficult to be distinguished or localized. This limitation also affects the efficiency of localization of those single-particle-localization microcopies, such as stochastic optical reconstruction microscopy (STORM) and (fluorescence) photoactivated localization microscopy [(F)PALM]. In this work, we developed a 3D sparse recovery (3D-SR) method, with the aim of localizing particles with high density in three dimensions, which cannot be resolved using original STROM or (F)PALM. A cylindrical lens was introduced to a traditional wide-field microscope in order to form the 3D point spread function for 3D-SR. The performance of the 3D-SR method was evaluated using simulation. Simulated results demonstrated that, even for particle densities as high as 4 μm-2 on a transversal projection, particles could still be localized with high accuracy. The standard deviations were found to be 25.59nm along the transverse direction and 50.42nm along the axial direction. Compared with the existing 3D localization methods used in high particle density cases, such as 3D-DAOSTORM, 3D-SR allows a higher activated fluorophore density per frame.

CALIBRATION OF THE NuSTAR HIGH-ENERGY FOCUSING X-RAY TELESCOPE

We present the calibration of the \textit{Nuclear Spectroscopic Telescope Array} (\nustar) X-ray satellite. We used the Crab as the primary effective area calibrator and constructed a piece-wise linear spline function to modify the vignetting response. The achieved residuals for all off-axis angles and energies, compared to the assumed spectrum, are typically better than $\pm 2$\% up to 40\,keV and 5--10\,\% above due to limited counting statistics. An empirical adjustment to the theoretical 2D point spread function (PSF) was found using several strong point sources, and no increase of the PSF half power diameter (HPD) has been observed since the beginning of the mission. We report on the detector gain calibration, good to 60\,eV for all grades, and discuss the timing capabilities of the observatory, which has an absolute timing of $\pm$ 3\,ms. Finally we present cross-calibration results from two campaigns between all the major concurrent X-ray observatories (\textit{Chandra}, \textit{Swift}, \textit{Suzaku} and \textit{XMM-Newton}), conducted in 2012 and 2013 on the sources 3C\,273 and PKS\,2155-304, and show that the differences in measured flux is within $\sim$10\% for all instruments with respect to \nustar.

Out-of-focus background subtraction for fast structured illumination super-resolution microscopy of optically thick samples.

We propose a structured illumination microscopy method to combine super resolution and optical sectioning in three-dimensional (3D) samples that allows the use of two-dimensional (2D) data processing. Indeed, obtaining super-resolution images of thick samples is a difficult task if low spatial frequencies are present in the in-focus section of the sample, as these frequencies have to be distinguished from the out-of-focus background. A rigorous treatment would require a 3D reconstruction of the whole sample using a 3D point spread function and a 3D stack of structured illumination data. The number of raw images required, 15 per optical section in this case, limits the rate at which high-resolution images can be obtained. We show that by a succession of two different treatments of structured illumination data we can estimate the contrast of the illumination pattern and remove the out-of-focus content from the raw images. After this cleaning step, we can obtain super-resolution images of optical sections in thick samples using a two-beam harmonic illumination pattern and a limited number of raw images. This two-step processing makes it possible to obtain super resolved optical sections in thick samples as fast as if the sample was two-dimensional.

3D multifocus astigmatism and compressed sensing (3D MACS) based superresolution reconstruction.

Single molecule based superresolution techniques (STORM/PALM) achieve nanometer spatial resolution by integrating the temporal information of the switching dynamics of fluorophores (emitters). When emitter density is low for each frame, they are located to the nanometer resolution. However, when the emitter density rises, causing significant overlapping, it becomes increasingly difficult to accurately locate individual emitters. This is particularly apparent in three dimensional (3D) localization because of the large effective volume of the 3D point spread function (PSF). The inability to precisely locate the emitters at a high density causes poor temporal resolution of localization-based superresolution technique and significantly limits its application in 3D live cell imaging. To address this problem, we developed a 3D high-density superresolution imaging platform that allows us to precisely locate the positions of emitters, even when they are significantly overlapped in three dimensional space. Our platform involves a multi-focus system in combination with astigmatic optics and an ℓ 1-Homotopy optimization procedure. To reduce the intrinsic bias introduced by the discrete formulation of compressed sensing, we introduced a debiasing step followed by a 3D weighted centroid procedure, which not only increases the localization accuracy, but also increases the computation speed of image reconstruction. We implemented our algorithms on a graphic processing unit (GPU), which speeds up processing 10 times compared with central processing unit (CPU) implementation. We tested our method with both simulated data and experimental data of fluorescently labeled microtubules and were able to reconstruct a 3D microtubule image with 1000 frames (512×512) acquired within 20 seconds.

Separable Coded Aperture for Depth from a Single Image

We propose the use of a separable coded aperture to estimate depth from a single defocused image, and derive a criterion for evaluating aperture patterns with respect to depth discrimination. With a separable coded aperture, two-dimensional (2D) point spread functions (PSFs) are separated into the product of horizontal and vertical one-dimensional (1D) PSFs. Depth is recovered by finding the scale of the 1D PSFs at each pixels based on the space-frequency analysis. Benefiting from 1D PSFs and the optimal code, our approach obtains more accurate depth especially on depth boundaries, and is faster since 2D computation is reduced to 1D. Experimental results on synthetic and real-captured data demonstrates the effectiveness of our approach.

Vectorial point spread function and optical transfer function in oblique plane imaging.

Oblique plane imaging, using remote focusing with a tilted mirror, enables direct two-dimensional (2D) imaging of any inclined plane of interest in three-dimensional (3D) specimens. It can image real-time dynamics of a living sample that changes rapidly or evolves its structure along arbitrary orientations. It also allows direct observations of any tilted target plane in an object of which orientational information is inaccessible during sample preparation. In this work, we study the optical resolution of this innovative wide-field imaging method. Using the vectorial diffraction theory, we formulate the vectorial point spread function (PSF) of direct oblique plane imaging. The anisotropic lateral resolving power caused by light clipping from the tilted mirror is theoretically analyzed for all oblique angles. We show that the 2D PSF in oblique plane imaging is conceptually different from the inclined 2D slice of the 3D PSF in conventional lateral imaging. Vectorial optical transfer function (OTF) of oblique plane imaging is also calculated by the fast Fourier transform (FFT) method to study effects of oblique angles on frequency responses.

Testing the accuracy of the two-dimensional object model in HAADF STEM

In high-angle annular dark-field scanning transmission electron microscopy (HAADF-STEM) the two-dimensional (2D) object-function approximation and image convolution model is often used to describe the recorded image data from on-axis crystalline samples. In this model the sample, represented by an object function with sharp peaks at the atomic column positions, is convolved with the 2D point spread function (PSF) of the focussed STEM illumination. In this work the validity of the assumption that the object can be considered as 2D is evaluated experimentally through the use of HAADF-STEM focal-series from MgO smoke cubes. The intensity of laterally resolved image-information is evaluated using Fourier transforms and is tracked with respect to defocus. The experimental results are compared with the expected depth resolution capabilities of simulated STEM probes to yield the normalised or ‘apparent thickness’ of the samples. The 2D object-function and image convolution models are found to hold for sample thicknesses of up to 250nm. As the 2D object model holds true for each individual frame in a recorded focal series, we can hence express the focal-series as a whole as the convolution of a 2D object function with a 3D probe function with implications for both the diagnosis of remnant aberrations and also image reconstruction.

A Multi-Emitter Localization Comparison of 3D Superresolution Imaging Modalities

Single-molecule localization-based superresolution imaging is complicated by emission from multiple emitters overlapping at the detector. The potential for overlapping emitters is even greater for 3D imaging than for 2D imaging due to the large effective ‘volume’ of the 3D point spread function (PSF). Overlapping emission can be accounted for in the estimation model, recovering the ability to localize the emitters, but with the caveat that the localization precision has a dependence on the amount of overlap from other emitters. It is interesting to consider if a particular 3D imaging modality has a significant advantage in facilitating the position estimation of overlapping emitters. We compare variants of two commonly used and easily implemented imaging modalities for 3D single-molecule imaging: astigmatic imaging; dual focal plane imaging; and the combination of the two approaches− dual focal plane with astigmatism.We quantify the ability for a particular 3D modality to facilitate estimation with overlapping emitters using the Cramer-Rao lower bound (CRLB). The CRLB is used to calculate the theoretical best localization precision under a multi-emitter estimation model. We investigate the performance of these 3D modalities under a wide range of conditions including various distributions of collected photons per frame, background counts, pixel sizes, and camera read-out noise values.

Depth-variant blind restoration with pupil-phase constraints for 3D confocal microscopy

Three-dimensional images of confocal laser scanning microscopy suffer from a depth-variant blur, due to refractive index mismatch between the different mediums composing the system as well as the specimen, leading to optical aberrations. Our goal is to develop an image restoration method for 3D confocal microscopy taking into account the blur variation with depth. The difficulty is that optical aberrations depend on the refractive index of the biological specimen. The depth-variant blur function or the Point Spread Function (PSF) is thus different for each observation. A blind or semi-blind restoration method needs to be developed for this system. For that purpose, we use a previously developed algorithm for the joint estimation of the specimen function (original image) and the 3D PSF, the continuously depth-variant PSF is approximated by a convex combination of a set of space-invariant PSFs taken at different depths. We propose to add to that algorithm a pupil-phase constraint for the PSF estimation, given by the the optical instrument geometry. We thus define a blind estimation algorithm by minimizing a regularized criterion in which we integrate the Gerchberg-Saxton algorithm allowing to include these physical constraints. We show the efficiency of this method relying on some numerical tests.

Three‐dimensional structure of the Z‐ring as a random network of FtsZ filaments

The spatial organization of the Z-ring, the central element of the bacterial division machinery, is not yet fully understood. Using optical tweezers and subpixel image analysis, we have recently shown that the radial width of the Z-ring in unconstricted Escherichia coli is about 100 nm. The relatively large width is consistent with the observations of others. Moreover, simulation of the experimental FtsZ distribution using the theoretical three-dimensional (3D) point spread function was strongly in favour of a toroidal rather than a thin cylindrical model of the Z-ring. Here, we show that the low density of FtsZ filaments in the ring coincides within experimental uncertainty with the critical density of a 3D random network of cylindrical sticks. This suggests that the Z-ring may consist of a percolating network of FtsZ filaments. Several factors that are expected to affect the polymerization state and the extent of self-interaction of FtsZ within the Z-ring, as well as the functional implications of its sparse toroidal structure, are discussed in terms of percolation theory.

Quantitative electron tomography: The effect of the three-dimensional point spread function

The intensity levels in a three-dimensional (3D) reconstruction, obtained by electron tomography, can be influenced by several experimental imperfections. Such artifacts will hamper a quantitative interpretation of the results. In this paper, we will correct for artificial intensity variations by determining the 3D point spread function (PSF) of a tomographic reconstruction based on high angle annular dark field scanning transmission electron microscopy. The large tails of the PSF cause an underestimation of the intensity of smaller particles, which in turn hampers an accurate radius estimate. Here, the error introduced by the PSF is quantified and corrected a posteriori.

Imaging characteristics of Zernike and annular polynomial aberrations

The general equations for the point-spread function (PSF) and optical transfer function (OTF) are given for any pupil shape, and they are applied to optical imaging systems with circular and annular pupils. The symmetry properties of the PSF, the real and imaginary parts of the OTF, and the modulation transfer function (MTF) of a system with a circular pupil aberrated by a Zernike circle polynomial aberration are derived. The interferograms and PSFs are illustrated for some typical polynomial aberrations with a sigma value of one wave, and 3D PSFs and MTFs are shown for 0.1 wave. The Strehl ratio is also calculated for polynomial aberrations with a sigma value of 0.1 wave, and shown to be well estimated from the sigma value. The numerical results are compared with the corresponding results in the literature. Because of the same angular dependence of the corresponding annular and circle polynomial aberrations, the symmetry properties of systems with annular pupils aberrated by an annular polynomial aberration are the same as those for a circular pupil aberrated by a corresponding circle polynomial aberration. They are also illustrated with numerical examples.

Speckle reduction of reconstructions of digital holograms using three dimensional filtering

We report on a new digital signal processing technique that reduces speckle in reconstructions of digital holograms. This is achieved by convolving the three dimensional intensity pattern (the intensity of the propagated DH at a series of different distances) with a 3D point spread function in all three dimensions (x,y,z). It is based on the fact that the addition of different independent speckle images on an intensity basis reduces the speckle content. We provide quantitative results in terms of speckle index and resolution, and show that filtering in the z direction has the added benefit of an increase in the depth of focus of the digital hologram reconstruction.