Complex Point Spread Function Research Articles

Semi-blind hyperspectral and multispectral image fusion via generalized inverse matrix optimization

Combining a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI) is an economical way to obtain a high spatial resolution hyperspectral image (HR-HSI). However, existing fusion methods often rely on a critical variable, the point spread function (PSF), which is difficult to obtain in practice. Additionally, these methods often require a long fusion time and have many parameters that need re-debugging on different datasets to achieve satisfactory results. Many existing methods lack robustness in handling complex scenarios, such as complex PSF, changing spatial downsampling factors, and large-scale datasets. These issues can result in degraded performance or render the methods inapplicable to practical scenarios. Therefore, we propose an efficient and parameterless semi-blind fusion method based on generalized inverse matrix optimization (GIMO), which is robust for multiple complex situations. Our method uses the definition of generalized inverse matrix and the relationship among HSI, MSI, and HR-HSI to approximate the function module of PSF. Then, we calculate the generalized inverse matrix of the spectral response function (SRF). Finally, we optimize the generalized inverse matrix of SRF using the Sylvester equation and multiply it with MSI to obtain HR-HSI. This is the first proposed method to achieve hyperspectral image super-resolution by optimizing the inverse matrix of the spectral response function. Experiments on multiple simulated datasets and one real dataset have validated that our GIMO method outperforms existing state-of-the-art methods, especially regarding fusion time and robustness for various complex scenarios.

Interactive simulation and visualization of point spread functions in single molecule imaging

The point spread function (PSF) is fundamental to any type of microscopy, most importantly so for single-molecule localization techniques, where the exact PSF shape is crucial for precise molecule localization at the nanoscale. Optical aberrations and fixed fluorophore dipoles often result in non-isotropic and distorted PSFs, impairing and biasing conventional fitting approaches. Further, PSF shapes are deliberately modified in PSF engineering approaches for providing improved sensitivity, e.g., for 3D localization or determination of dipole orientation. As this can lead to highly complex PSF shapes, a tool for visualizing expected PSFs would facilitate the interpretation of obtained data and the design of experimental approaches. To this end, we introduce a comprehensive and accessible computer application that allows for the simulation of realistic PSFs based on the full vectorial PSF model. Our tool incorporates a wide range of microscope and fluorophore parameters, including orientationally constrained fluorophores, as well as custom aberrations, transmission and phase masks, thus enabling an accurate representation of various imaging conditions. An additional feature is the simulation of crowded molecular environments with overlapping PSFs. Further, our app directly provides the Cramér–Rao bound for assessing the best achievable localization precision under given conditions. Finally, our software allows for the fitting of custom aberrations directly from experimental data, as well as the generation of a large dataset with randomized simulation parameters, effectively bridging the gap between simulated and experimental scenarios, and enhancing experimental design and result validation.

High-density volumetric super-resolution microscopy

Volumetric super-resolution microscopy typically encodes the 3D position of single-molecule fluorescence into a 2D image by changing the shape of the point spread function (PSF) as a function of depth. However, the resulting large and complex PSF spatial footprints reduce biological throughput and applicability by requiring lower labeling densities to avoid overlapping fluorescent signals. We quantitatively compare the density dependence of single-molecule light field microscopy (SMLFM) to other 3D PSFs (astigmatism, double helix and tetrapod) showing that SMLFM enables an order-of-magnitude speed improvement compared to the double helix PSF by resolving overlapping emitters through parallax. We demonstrate this optical robustness experimentally with high accuracy ( > 99.2 ± 0.1%, 0.1 locs μm−2) and sensitivity ( > 86.6 ± 0.9%, 0.1 locs μm−2) through whole-cell (scan-free) imaging and tracking of single membrane proteins in live primary B cells. We also exemplify high-density volumetric imaging (0.15 locs μm−2) in dense cytosolic tubulin datasets.

Deconvolution of JWST/MIRI Images: Applications to an Active Galactic Nucleus Model and GATOS Observations of NGC 5728

The superb image quality, stability, and sensitivity of JWST permit deconvolution techniques to be pursued with a fidelity unavailable to ground-based observations. We present an assessment of several deconvolution approaches to improve image quality and mitigate the effects of the complex JWST point-spread function (PSF). The optimal deconvolution method is determined by using WebbPSF to simulate JWST’s complex PSF and MIRISim to simulate multiband JWST/Mid-Infrared Imager Module (MIRIM) observations of a toy model of an active galactic nucleus (AGN). Five different deconvolution algorithms are tested: (1) Kraken deconvolution, (2) Richardson–Lucy, (3) the adaptive imaging deconvolution algorithm, (4) sparse regularization with the Condat–Vũ algorithm, and (5) iterative Wiener filtering and thresholding. We find that Kraken affords the greatest FWHM reduction of the nuclear source of our MIRISim observations for the toy AGN model while retaining good photometric integrity across all simulated wave bands. Applying Kraken to Galactic Activity, Torus, and Outflow Survey (GATOS) multiband JWST/MIRIM observations of the Seyfert 2 galaxy NGC 5728, we find that the algorithm reduces the FWHM of the nuclear source by a factor of 1.6–2.2 across all five filters. Kraken images facilitate detection of extended nuclear emission ∼2.″5 (∼470 pc, position angle ≃ 115°) in the SE–NW direction, especially at the longest wavelengths. We demonstrate that Kraken is a powerful tool to enhance faint features otherwise hidden in the complex JWST PSF.

Single-pixel three-dimensional imaging of the terahertz-wave by complex-field synthesis.

We propose a novel three-dimensional (3D) imaging technique by terahertz (THz) waves. Specifically, we modulate the THz wave using diffusers to produce three different speckle-like illumination patterns. The object is raster scanned by the three illumination patterns to generate three raw images via the single-pixel detection method. Subsequently, we synthesize a complex field using the three raw images. Finally, the retrieved image is calculated using the phase correlation of the complex point spread function. The proposed imaging system is simple and highly cost-effective. Therefore, it is a promising technique that can be adopted for industrial inspection and security screening.

Three-dimensional imaging by interferenceless optical scanning holography

Optical scanning holography (OSH) can perform high quality, speckle-free three-dimensional (3D) holographic imaging. However, conventional OSH system contains an interferometer, which is sensitive to environmental disturbances. Here, we develop an interferenceless OSH (IOSH) system to overcome this issue. In IOSH, the interferometer is replaced by a digital micro device (DMD) together with a projection module. The Fresnel zone plates are directly displayed on the DMD and projected to the object target. The high frame-rate DMD can serve as both the display device and the scanning device, and thus the system can be compact, robust, and low cost. We have studied the system performance by analyzing the 3D complex point spread function. The ability of 3D imaging by IOSH has also been demonstrated experimentally.

Deconvolution-based distortion correction of EPI using analytic single-voxel point-spread functions.

To develop a postprocessing algorithm that corrects geometric distortions due to spatial variations of the static magnetic field amplitude, B0 , and effects from relaxation during signal acquisition in EPI. An analytic, complex point-spread function is deduced for k-space trajectories of EPI variants and applied to corresponding acquisitions in a resolution phantom and in human volunteers at 3 T. With the analytic point-spread function and experimental maps of B0 (and, optionally, the effective transverse relaxation time, ) as input, a point-spread function matrix operator is devised for distortion correction by a Thikonov-regularized deconvolution in image space. The point-spread function operator provides additional information for an appropriate correction of the signal intensity distribution. A previous image combination algorithm for acquisitions with opposite phase blip polarities is adapted to the proposed method to recover destructively interfering signal contributions. Applications of the proposed deconvolution-based distortion correction ("DecoDisCo") algorithm demonstrate excellent distortion corrections and superior performance regarding the recovery of an undistorted intensity distribution in comparison to a multifrequency reconstruction. Examples include full and partial Fourier standard EPI scans as well as double-shot center-out trajectories. Compared with other distortion-correction approaches, DecoDisCo permits additional deblurring to obtain sharper images in cases of significant effects. Robust distortion corrections in EPI acquisitions are feasible with high quality by regularized deconvolution with an analytic point-spread function. The general algorithm, which is publicly released on GitHub, can be straightforwardly adapted for specific EPI variants or other acquisition schemes.

Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet

Background fluorescence, especially when it exhibits undesired spatial features, is a primary factor for reduced image quality in optical microscopy. Structured background is particularly detrimental when analyzing single-molecule images for 3-dimensional localization microscopy or single-molecule tracking. Here, we introduce BGnet, a deep neural network with a U-net-type architecture, as a general method to rapidly estimate the background underlying the image of a point source with excellent accuracy, even when point-spread function (PSF) engineering is in use to create complex PSF shapes. We trained BGnet to extract the background from images of various PSFs and show that the identification is accurate for a wide range of different interfering background structures constructed from many spatial frequencies. Furthermore, we demonstrate that the obtained background-corrected PSF images, for both simulated and experimental data, lead to a substantial improvement in localization precision. Finally, we verify that structured background estimation with BGnet results in higher quality of superresolution reconstructions of biological structures.

Physics-based model of the adaptive-optics-corrected point spread function

Context.Adaptive optics (AO) systems greatly increase the resolution of large telescopes, but produce complex point spread function (PSF) shapes, varying in time and across the field of view. The PSF must be accurately known since it provides crucial information about optical systems for design, characterization, diagnostics, and image post-processing.Aims.We develop here a model of the AO long-exposure PSF, adapted to various seeing conditions and any AO system. This model is made to match accurately both the core of the PSF and its turbulent halo.Methods.The PSF model we develop is based on a parsimonious parameterization of the phase power spectral density, with only five parameters to describe circularly symmetric PSFs and seven parameters for asymmetrical ones. Moreover, one of the parameters is the Fried parameterr0of the turbulence’s strength. This physical parameter is an asset in the PSF model since it can be correlated with external measurements of ther0, such as phase slopes from the AO real time computer (RTC) or site seeing monitoring.Results.We fit our model against end-to-end simulated PSFs using the OOMAO tool, and against on-sky PSFs from the SPHERE/ZIMPOL imager and the MUSE integral field spectrometer working in AO narrow-field mode. Our model matches the shape of the AO PSF both in the core and the halo, with a relative error smaller than 1% for simulated and experimental data. We also show that we retrieve ther0parameter with sub-centimeter precision on simulated data. For ZIMPOL data, we show a correlation of 97% between ourr0estimation and the RTC estimation. Finally, MUSE allows us to test the spectral dependency of the fittedr0parameter. It follows the theoreticalλ6/5evolution with a standard deviation of 0.3 cm. Evolution of other PSF parameters, such as residual phase variance or aliasing, is also discussed.

Signal enhanced holographic fluorescence microscopy with guide-star reconstruction.

We propose a signal enhanced guide-star reconstruction method for holographic fluorescence microscopy. In the late 00's, incoherent digital holography started to be vigorously studied by several groups to overcome the limitations of conventional digital holography. The basic concept of incoherent digital holography is to acquire the complex hologram from incoherent light by utilizing temporal coherency of a spatially incoherent light source. The advent of incoherent digital holography opened new possibility of holographic fluorescence microscopy (HFM), which was difficult to achieve with conventional digital holography. However there has been an important issue of low and noisy signal in HFM which slows down the system speed and degrades the imaging quality. When guide-star reconstruction is adopted, the image reconstruction gives an improved result compared to the conventional propagation reconstruction method. The guide-star reconstruction method gives higher imaging signal-to-noise ratio since the acquired complex point spread function provides optimal system-adaptive information and can restore the signal buried in the noise more efficiently. We present theoretical explanation and simulation as well as experimental results.

Holographic fluorescence microscopy with incoherent digital holographic adaptive optics.

Introduction of adaptive optics technology into astronomy and ophthalmology has made great contributions in these fields, allowing one to recover images blurred by atmospheric turbulence or aberrations of the eye. Similar adaptive optics improvement in microscopic imaging is also of interest to researchers using various techniques. Current technology of adaptive optics typically contains three key elements: a wavefront sensor, wavefront corrector, and controller. These hardware elements tend to be bulky, expensive, and limited in resolution, involving, for example, lenslet arrays for sensing or multiactuator deformable mirrors for correcting. We have previously introduced an alternate approach based on unique capabilities of digital holography, namely direct access to the phase profile of an optical field and the ability to numerically manipulate the phase profile. We have also demonstrated that direct access and compensation of the phase profile are possible not only with conventional coherent digital holography, but also with a new type of digital holography using incoherent light: selfinterference incoherent digital holography (SIDH). The SIDH generates a complex—i.e., amplitude plus phase—hologram from one or several interferograms acquired with incoherent light, such as LEDs, lamps, sunlight, or fluorescence. The complex point spread function can be measured using guide star illumination and it allows deterministic deconvolution of the full-field image. We present experimental demonstration of aberration compensation in holographic fluorescence microscopy using SIDH. Adaptive optics by SIDH provides new tools for improved cellular fluorescence microscopy through intact tissue layers or other types of aberrant media.

GREAT3 results – I. Systematic errors in shear estimation and the impact of real galaxy morphology

We present first results from the third GRavitational lEnsing Accuracy Testing (GREAT3) challenge, the third in a sequence of challenges for testing methods of inferring weak gravitational lensing shear distortions from simulated galaxy images. GREAT3 was divided into experiments to test three specific questions, and included simulated space- and ground-based data with constant or cosmologically-varying shear fields. The simplest (control) experiment included parametric galaxies with a realistic distribution of signal-to-noise, size, and ellipticity, and a complex point spread function (PSF). The other experiments tested the additional impact of realistic galaxy morphology, multiple exposure imaging, and the uncertainty about a spatially-varying PSF; the last two questions will be explored in Paper II. The 24 participating teams competed to estimate lensing shears to within systematic error tolerances for upcoming Stage-IV dark energy surveys, making 1525 submissions overall. GREAT3 saw considerable variety and innovation in the types of methods applied. Several teams now meet or exceed the targets in many of the tests conducted (to within the statistical errors). We conclude that the presence of realistic galaxy morphology in simulations changes shear calibration biases by $\sim 1$ per cent for a wide range of methods. Other effects such as truncation biases due to finite galaxy postage stamps, and the impact of galaxy type as measured by the S\'{e}rsic index, are quantified for the first time. Our results generalize previous studies regarding sensitivities to galaxy size and signal-to-noise, and to PSF properties such as seeing and defocus. Almost all methods' results support the simple model in which additive shear biases depend linearly on PSF ellipticity.

On the complex point spread function in interferometric cross-polarisation microscopy.

The ability to characterize the Point Spread Function(PSF) is crucial in practical microscopy, but requires knowledge of the complex PSF for approaches that detect fields instead of intensities. Here we experimentally measure and theoretically model the volumetric amplitude and phase response of an Interferometric Cross-polarisation Microscope to demonstrate the technique's capability to provide confocal-like images of weakly birefringent structures in living cells. We find the axial FWHM of the amplitude PSF to be 0.70 ± 0.01 μm and 0.83 μm for model and measurement, respectively, on par with confocal microscopy. Ultimately retaining both amplitude and phase information will however enable approaches for improved localisation of objects.

Determination of the point spread function of layered metamaterials assisted with the blind deconvolution algorithm

We propose to use the blind deconvolution and its modification to extract the point spread function (PSF) of layered metamaterials from a SNOM measurement. The measurement results are processed using the blind deconvolution algorithm to reconstruct the real-valued non-coherent PSF, or using the modified blind deconvolution introduced in this paper to reconstruct the complex-valued coherent PSF. The two algorithms are tested in simulations with a layered metamaterial deposited on a thin metallic mask with test apertures. We show that the modified algorithm is capable of recovering the approximate shape of complex PSF with a sub-wavelength full width at half maximum from a measurement in which the apertures are larger than the wavelength.

Atmospheric point spread function interpolation for weak lensing in short exposure imaging data

A main science goal for the Large Synoptic Survey Telescope (LSST) is to measure the cosmic shear signal from weak lensing to extreme accuracy. One difficulty, however, is that with the short exposure time ($\simeq$15 seconds) proposed, the spatial variation of the Point Spread Function (PSF) shapes may be dominated by the atmosphere, in addition to optics errors. While optics errors mainly cause the PSF to vary on angular scales similar or larger than a single CCD sensor, the atmosphere generates stochastic structures on a wide range of angular scales. It thus becomes a challenge to infer the multi-scale, complex atmospheric PSF patterns by interpolating the sparsely sampled stars in the field. In this paper we present a new method, PSFent, for interpolating the PSF shape parameters, based on reconstructing underlying shape parameter maps with a multi-scale maximum entropy algorithm. We demonstrate, using images from the LSST Photon Simulator, the performance of our approach relative to a 5th-order polynomial fit (representing the current standard) and a simple boxcar smoothing technique. Quantitatively, PSFent predicts more accurate PSF models in all scenarios and the residual PSF errors are spatially less correlated. This improvement in PSF interpolation leads to a factor of 3.5 lower systematic errors in the shear power spectrum on scales smaller than $\sim13'$, compared to polynomial fitting. We estimate that with PSFent and for stellar densities greater than $\simeq1/{\rm arcmin}^{2}$, the spurious shear correlation from PSF interpolation, after combining a complete 10-year dataset from LSST, is lower than the corresponding statistical uncertainties on the cosmic shear power spectrum, even under a conservative scenario.

Determining the complex point-spread function of a scanning differential heterodyne microscope

For a scanning differential heterodyne microscope, the two-dimensional point spread function and the pupil function are determined from experimental measurements of the intensity distribution of a probing beam. The phase components of these functions were restored from the measured distributions using the Gerchberg-Saxton algorithm. Based on the two-dimensional pupil function, the three-dimensional point-spread function is determined in the scalar approximation using the Debye integral. The aberration function of the microscope is analyzed from the viewpoint of the composition of aberrations and method of their balancing. The focusing adjustment criterion of the microscope for an object under study by the phase response from the test structure is considered.

Iterative PSF Estimation and Its Application to Shift Invariant and Variant Blur Reduction

Among image restoration approaches, image deconvolution has been considered a powerful solution. In image deconvolution, a point spread function (PSF), which describes the blur of the image, needs to be determined. Therefore, in this paper, we propose an iterative PSF estimation algorithm which is able to estimate an accurate PSF. In real-world motion-blurred images, a simple parametric model of the PSF fails when a camera moves in an arbitrary direction with an inconsistent speed during an exposure time. Moreover, the PSF normally changes with spatial location. In order to accurately estimate the complex PSF of a real motion blurred image, we iteratively update the PSF by using a directional spreading operator. The directional spreading is applied to the PSF when it reduces the amount of the blur and the restoration artifacts. Then, to generalize the proposed technique to the linear shift variant (LSV) model, a piecewise invariant approach is adopted by the proposed image segmentation method. Experimental results show that the proposed method effectively estimates the PSF and restores the degraded images.

Estimation of a super-resolved PSF for the data reduction of undersampled stellar observations

Context. Fitting photometry algorithms can be very effective provided that an accurate model of the instrumental point spread function (PSF) is available. When high-precision time-resolved photometry is required, however, the use of point-source star images as empirical PSF models can be unsatisfactory, due to the limits in their spatial resolution. Theoretically-derived models, on the other hand, are limited by the unavoidable assumption of simplifying hypothesis, while the use of analytical approximations is restricted to regularly-shaped PSFs. Aims. This work investigates an innovative technique for space-based fitting photometry, based on the reconstruction of an empirical but properly-resolved PSF. The aim is the exploitation of arbitrary star images, including those produced under intentional defocus. The cases of both MOST and COROT, the first space telescopes dedicated to time-resolved stellar photometry, are considered in the evaluation of the effectiveness and performances of the proposed methodology. Methods. PSF reconstruction is based on a set of star images, periodically acquired and presenting relative subpixel displacements due to motion of the acquisition system, in this case the jitter of the satellite attitude. Higher resolution is achieved through the solution of the inverse problem. The approach can be regarded as a special application of super-resolution techniques, though a specialised procedure is proposed to better meet the PSF determination problem specificities. The application of such a model to fitting photometry is illustrated by numerical simulations for COROT and on a complete set of observations from MOST. Results. We verify that, in both scenarios, significantly better resolved PSFs can be estimated, leading to corresponding improvements in photometric results. For COROT, indeed, subpixel reconstruction enabled the successful use of fitting algorithms despite its rather complex PSF profile, which could hardly be modeled otherwise. For MOST, whose direct-imaging PSF is closer to the ordinary, comparison to other models or photometry techniques were carried out and confirmed the potential of PSF reconstruction in real observational conditions.

Phase retrieval using coherent imaging systems with linear transfer functions

We consider the problem of quantitative phase retrieval from images obtained using a coherent shift-invariant linear imaging system whose associated transfer function (i.e., the Fourier transform of the complex point-spread function) is well approximated by a linear function of spatial frequency. This linear approximation to the transfer function is applicable when the spread of spatial frequencies, in a two-dimensional complex wavefield, is sufficiently narrow when compared to the characteristic length of variation of the transfer function for an imaging system taking such a wavefield as input. We give several algorithms for reconstructing both the phase and amplitude of a given two-dimensional coherent wavefield, given as input data one or more images of such a wavefield which may be formed by different states of the imaging system. When an object to be imaged consists of a single material, or of a single material embedded in a substrate of constant thickness, the phase-amplitude reconstruction can be performed using a single image. As a first application of these ideas, we write down an algorithm for using a single diffraction-enhanced image (DEI) to obtain a quantitative reconstruction of the projected thickness of a single-material sample which is embedded within a substrate of approximately constant thickness. This algorithm is used to quantitatively map inclusions in a breast phantom, from a single synchrotron DEI image of the same. In particular, the reconstructed images quantitatively represent the projected thickness in the bulk of the sample, in contrast to raw DEI images which greatly emphasise sharp edges (high spatial frequencies). Lastly, we point out that the methods presented here are also applicable to the quantitative analysis of differential interference contrast (DIC) images, obtained using both visible-light and X-ray microscopy.

From Scattering to Wavefronts-What's in Between?

Some optical errors are too localized and random to be detected by commercial wavefront devices and Zernike polynomial expression. We looked beyond aberrations defined by Zernike expression to discuss implications of fine irregularities associated with highly aberrated corneal surfaces and complex surface roughness that can lead to light scattering. Most fine irregularities are related to postoperative surface roughness, complexities of corneal ablation, and the laser in situ keratomileusis (LASIK) flap. These can be characterized mathematically by a random function that includes local surface tilts, the correlation radius of irregularities (Ic), surface roughness, and other terms. The Kirchoff method of scatter analysis characterizes fine surface irregularities by replacing each point on the surface with a tangential plane, allowing it to be governed by Snellen and Fresnel laws. The joint action of the continuum of microbeams defines a complex point spread function that can be expressed by the Strehl ratio. Small, highly irregular steep central islands and flap striae may not be adequately detected by Zernike expression and may have a surface irregularity diameter of 0.1 to 2.0 mm and height of 10 to 20 microm that results in a reduced Strehl ratio below 0.8. Laser ablation inhomogeneities may have dimensions of 1 to 10 microm, resulting in a root mean square tilt value approaching 1.0 and a Strehl ratio below 0.5. Corneal surface irregularities after laser vision correction may induce significant optical aberrations and distortions apart from classical wavefront or scattering errors. As these may not be detected by commercial wavefront devices, and yet contribute to the degradation of optical performance, alternate techniques should be evaluated to detect and describe these surface irregularities.