Variant Point Spread Function Research Articles

Hybrid Space Calibrated 3D Network of Diffractive Hyperspectral Optical Imaging Sensor.

Diffractive multispectral optical imaging plays an essential role in optical sensing, which typically suffers from the image blurring problem caused by the spatially variant point spread function. Here, we propose a novel high-quality and efficient hybrid space calibrated 3D network "HSC3D" for spatially variant diffractive multispectral imaging that utilizes the 3D U-Net structure combined with space calibration modules of magnification and rotation effects to achieve high-accuracy eight-channel multispectral restoration. The algorithm combines the advantages of the space calibrated module and U-Net architecture with 3D convolutional layers to improve the image quality of diffractive multispectral imaging without the requirements of complex equipment modifications and large amounts of data. A diffractive multispectral imaging system is established by designing and manufacturing one diffractive lens and four refractive lenses, whose monochromatic aberration is carefully corrected to improve imaging quality. The mean peak signal-to-noise ratio and mean structural similarity index of the reconstructed multispectral images are improved by 3.33 dB and 0.08, respectively, presenting obviously improved image quality compared with a typical Unrolled Network algorithm. The new algorithm with high space calibrated ability and imaging quality has great application potential in diffraction lens spectroscopy and paves a new method for complex practical diffractive multispectral image sensing.

Star Image Centering with Deep Learning. II. HST/WFPC2 Full Field of View

We present an expanded and improved deep-learning (DL) methodology for determining centers of star images on Hubble Space Telescope/Wide-Field Planetary Camera 2 (WFPC2) exposures. Previously, we demonstrated that our DL model can eliminate the pixel-phase bias otherwise present in these undersampled images; however that analysis was limited to the central portion of each detector. In the current work we introduce the inclusion of global positions to account for the point-spread function (PSF) variation across the entire chip and instrumental magnitudes to account for nonlinear effects such as charge transfer efficiency. The DL model is trained using a unique series of WFPC2 observations of globular cluster 47 Tuc, data sets comprising over 600 dithered exposures taken in each of two filters—F555W and F814W. It is found that the PSF variations across each chip correspond to corrections of the order of ∼100 mpix, while magnitude effects are at a level of ∼10 mpix. Importantly, pixel-phase bias is eliminated with the DL model; whereas, with a classic centering algorithm, the amplitude of this bias can be up to ∼40 mpix. Our improved DL model yields star-image centers with uncertainties of 8–10 mpix across the full field of view of WFPC2.

Pushing the limits of near-infrared photometry with the Gemini Multi-Conjugate Adaptive Optics System: study of crowded fields in the globular cluster M5

ABSTRACT We present the deepest J −Ks near-infrared photometry of the globular cluster M5 (NGC 5904) from observations taken with the Gemini South Adaptive Optics Imager in tandem with the Gemini Multi-conjugate adaptive optics System (GeMS) on the 8.1-m Gemini South telescope. Point spread function (PSF) photometry was carried out using a spatially variable PSF, zero-point calibrations based on correlations to a standard photometric catalogue, colour corrections, and crowding corrections. The latter corrections provided a new challenge given the field variations of the adaptive optics corrections in the central crowded regions of this cluster. The final photometric precision in our J− Ks colour–magnitude diagram exposes a dispersion among the lower main-sequence stars of M5 for the first time. This dispersion occurs below a main-sequence knee due to variations in the helium and CNO (carbon, nitrogen, and oxygen) abundances from multiple stellar populations, consistent with results from the bright evolved stars in this cluster from ultraviolet to near-infrared Hubble Space Telescope photometry and ground-based spectroscopy. This paper completes our original GeMS quality analysis programme, providing insights into adaptive optics analyses in crowded fields.

Large-FOV 3D localization microscopy by spatially variant point spread function generation.

Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify the modeling of the imaging system. However, for large fields of view (FOV) imaging, it becomes important to account for the spatially variant nature of the PSF. Here, we propose an accurate and fast principal components analysis-based field-dependent 3D PSF generator (PPG3D) and localizer for LM. Through simulations and experimental three-dimensional (3D) single-molecule localization microscopy (SMLM), we demonstrate the effectiveness of PPG3D, enabling super-resolution imaging of mitochondria and microtubules with high fidelity over a large FOV. A comparison of PPG3D with a shift-variant PSF generator for 3D LM reveals a threefold improvement in accuracy. Moreover, PPG3D is approximately 100 times faster than existing PSF generators, when used in image plane-based interpolation mode. Given its user-friendliness, we believe that PPG3D holds great potential for widespread application in SMLM and other imaging modalities.

Weak gravitational lensing measurements of Abell 2744 using JWST and shear measurement algorithm pyRRG-JWST

ABSTRACT We update the publicly available weak lensing shear measurement algorithm pyRRG for the JWST, and apply it to UNCOVER DR1 imaging of galaxy cluster Abell 2744. At short wavelengths (${\lt}2.5\, \mu$m), shear measurements are consistent between independent observations through different JWST bandpasses, and calibrated within 1.5 per cent of those from the Hubble Space Telescope. At longer wavelengths, shear is underestimated by ∼5 per cent, probably due to coarser pixellization. We model the spatially varying point spread function using WebbPSF, whose moments are within 0.05 of real stars near the centre of the mosaic, where there are sufficient stars to also generate an empirical model. We measure shear from up to 162 galaxies arcmin−2 to derive a map of dark plus baryonic mass with 12 arcsec (55 kpc) spatial resolution. All code, catalogues, and maps are available from https://github.com/davidharvey1986/pyRRG.

Deconvolutional Method Utilizing Power Variable Point Spread Function for Resolution Degradation Mitigation in SBS-Based Optical Spectrum Analyzer

Deconvolutional Method Utilizing Power Variable Point Spread Function for Resolution Degradation Mitigation in SBS-Based Optical Spectrum Analyzer

Image Recovery for Low Earth Orbit by Leveraging Turbulence and Light Curves

This paper shows a novel method to characterize human-made objects in low Earth orbit (LEO) using compressed sensing on light curve measurements. The proposed approach minimizes total variation to recover a resolved object image from a fully unresolved light curve and a so-called point spread function (PSF) map. The light curves are generated through numerical wave propagation, which considers atmospheric turbulence under anisoplanatic conditions. Subsequently, the light curve model is transformed into a linear measurement model to apply compressed sensing techniques. Notably, the sensing matrix is found to be a superposition of spatially variable PSFs, which significantly downsamples the ideal object image. The proposed approach robustly recovers clear images of objects in LEO, even with imperfect PSF map estimates and Poisson noise in the light curve measurement.

Perception of misalignment states for sky survey telescopes with the digital twin and the deep neural networks.

Sky survey telescopes play a critical role in modern astronomy, but misalignment of their optical elements can introduce significant variations in point spread functions, leading to reduced data quality. To address this, we need a method to obtain misalignment states, aiding in the reconstruction of accurate point spread functions for data processing methods or facilitating adjustments of optical components for improved image quality. Since sky survey telescopes consist of many optical elements, they result in a vast array of potential misalignment states, some of which are intricately coupled, posing detection challenges. However, by continuously adjusting the misalignment states of optical elements, we can disentangle coupled states. Based on this principle, we propose a deep neural network to extract misalignment states from continuously varying point spread functions in different field of views. To ensure sufficient and diverse training data, we recommend employing a digital twin to obtain data for neural network training. Additionally, we introduce the state graph to store misalignment data and explore complex relationships between misalignment states and corresponding point spread functions, guiding the generation of training data from experiments. Once trained, the neural network estimates misalignment states from observation data, regardless of the impacts caused by atmospheric turbulence, noise, and limited spatial sampling rates in the detector. The method proposed in this paper could be used to provide prior information for the active optic system and the optical system alignment.

Resolution recovery on list mode MLEM reconstruction for dynamic cardiac SPECT system

The Dynamic Cardiac SPECT (DC-SPECT) system is being developed at the Massachusetts General Hospital, featuring a static cardio focus asymmetrical geometry enabling simultaneous high-resolution and high-sensitivity imaging. Among 14 design iterations of the DC-SPECT with varying number of detector heads, system sensitivity and resolution, the current version under development features 10 mm FWHM geometrical resolution (without resolution recovery) and 0.07% sensitivity at the center of the FOV, this is 1.5× resolution gain and 7× sensitivity gain compared to a conventional dual head gamma camera (0.01% sensitivity and 15-mm resolution). This work presents improvement in imaging resolution by implementing a spatially variant point spread function (SV-PSF) with list mode MLEM reconstruction. A resolution recovery method by PSF deconvolution is validated on list mode MLEM reconstruction for the DC-SPECT. A spatial invariant PSF is included as an additional test to show the influence of the PSF modelling accuracy on reconstructed image quality. We compare the MLEM reconstruction with and without PSF deconvolution; an analytic model is used for the calculation of system response, and the results are compared to the reconstruction with system modelling using Monte Carlo (MC) based methods. Results show that with PSF modelling applied, the quality of the reconstructed image is improved, and the DC-SPECT system can achieve a 4.5 mm central spatial resolution with average 795 counts/Mbq. Both the SV-PSF and the spatial-invariant PSF improve the image quality, and the reconstruction with SV-PSF generates line profiles closer to the ground truth. The results show substantial improvement over the GE Discovery 570c performance (7 mm spatial resolution with an average 460 counts/MBq, 5.8 mm resolution at the FOV center). The impact of PSF deconvolution is significant, improvement of the reconstructed image quality is evident in comparison to MC simulated system matrix with the same sampling size in the simulation.

A general framework for the assessment of scatter correction techniques in digital mammography

Scattered radiation negatively impacts radiographic imaging, with particular regard to mammography. In clinical practice, anti-scatter grids are exploited for this purpose; however, anti-scatter grids may also degrade the image quality, since they remove part of the useful primary radiation with the consequent increase of the dose to be administered to the patient. A suitable digital scatter correction method could tackle the limits imposed by such grids with a great impact on diagnosis. The main contribution of this study is the development of a general framework for the assessment of digital scatter correction techniques in mammography. To this aim, the formation process of both primary and scattered image is described on the basis of a systems-theory approach. Through a simulation of the radiological process, a reference model of the primary image is obtained and used as ground truth to compare the intensities of images obtained by applying a deconvolution-based digital scattering correction technique. Then, an experimental case study on breast phantom images is carried out to assess the scatter correction using different Point Spread Functions (PSFs) (Gaussian and Hyperbolic) with varying parameters values. A central issue was the identification of a spatially variant PSF to model the scattered radiation. The results demonstrate that the proposed approach enables the assessment and the comparison of different PSF kernels employed for scatter correction; in particular, our procedure shows that rather low relative errors are obtained ([−0.5;0.5]) for both the PSFs tested and that Gaussian ones are more sensitive to variations in their parameters.

Reconstruction of multi-animal PET acquisitions with anisotropically variant PSF

Among other factors such as random, attenuation and scatter corrections, uniform spatial resolution is key to performing accurate quantitative studies in Positron emission tomography (PET). Particularly in preclinical PET studies involving simultaneous acquisition of multiple animals, the degradation of image resolution due to the depth of interaction (DOI) effect far from the center of the Field of View (FOV) becomes a significant concern. In this work, we incorporated a spatially-variant resolution model into a real time iterative reconstruction code to obtain accurate images of multi-animal acquisition. We estimated the spatially variant point spread function (SV-PSF) across the FOV using measurements and Monte Carlo (MC) simulations. The SV-PSF obtained was implemented in a GPU-based Ordered subset expectation maximization (OSEM) reconstruction code, which includes scatter, attenuation and random corrections. The method was evaluated with acquisitions from two preclinical PET/CT scanners of the SEDECAL Argus family: a Derenzo phantom placed 2 cm off center in the 4R-SuperArgus, and a multi-animal study with 4 mice in the 6R-SuperArgus. The SV-PSF reconstructions showed uniform spatial resolution without significant increase in reconstruction time, with superior image quality compared to the uniform PSF model.

Image restoration for optical zooming system based on Alvarez lenses.

Alvarez lenses are known for their ability to achieve a broad range of optical power adjustment by utilizing complementary freeform surfaces. However, these lenses suffer from optical aberrations, which restrict their potential applications. To address this issue, we propose a field of view (FOV) attention image restoration model for continuous zooming. In order to simulate the degradation of optical zooming systems based on Alvarez lenses (OZA), a baseline OZA is designed where the polynomial for the Alvarez lenses consists of only three coefficients. By computing spatially varying point spread functions (PSFs), we simulate the degraded images of multiple zoom configurations and conduct restoration experiments. The results demonstrate that our approach surpasses the compared methods in the restoration of degraded images across various zoom configurations while also exhibiting strong generalization capabilities under untrained configurations.

Richardson–Lucy Deconvolution with a Spatially Variant Point-spread Function of Chandra: Supernova Remnant Cassiopeia A as an Example

Richardson–Lucy (RL) deconvolution is one of the classical methods widely used in X-ray astronomy and other areas. Amid recent progress in image processing, RL deconvolution still leaves much room for improvement under realistic situations. One direction is to include the positional dependence of a point-spread function (PSF), so-called RL deconvolution with a spatially variant PSF (RLsv). Another is the method of estimating a reliable number of iterations and their associated uncertainties. We developed a practical method that incorporates the RLsv algorithm and the estimation of uncertainties. As a typical example of bright and high-resolution images, the Chandra X-ray image of the supernova remnant Cassiopeia A was used in this paper. RLsv deconvolution enables us to uncover the smeared features in the forward/backward shocks and jet-like structures. We constructed a method to predict the appropriate number of iterations using statistical fluctuation of the observed images. Furthermore, the uncertainties were estimated by error propagation from the last iteration, which was phenomenologically tested with the observed data. Thus, our method is a practically efficient framework to evaluate the time evolution of the remnants and their fine structures embedded in high-resolution X-ray images.

Spatially-variant image deconvolution for photoacoustic tomography.

Photoacoustic tomography (PAT) system can reconstruct images of biological tissues with high resolution and contrast. However, in practice, the PAT images are usually degraded by spatially variant blur and streak artifacts due to the non-ideal imaging conditions and chosen reconstruction algorithms. Therefore, in this paper, we propose a two-phase restoration method to progressively improve the image quality. In the first phase, we design a precise device and measuring method to obtain spatially variant point spread function samples at preset positions of the PAT system in image domain, then we adopt principal component analysis and radial basis function interpolation to model the entire spatially variant point spread function. Afterwards, we propose a sparse logarithmic gradient regularized Richardson-Lucy (SLG-RL) algorithm to deblur the reconstructed PAT images. In the second phase, we present a novel method called deringing which is also based on SLG-RL to remove the streak artifacts. Finally, we evaluate our method with simulation, phantom and in vivo experiments, respectively. All the results show that our method can significantly improve the quality of PAT images.

Butterfly Transforms for Efficient Representation of Spatially Variant Point Spread Functions in Bayesian Imaging

Bayesian imaging algorithms are becoming increasingly important in, e.g., astronomy, medicine and biology. Given that many of these algorithms compute iterative solutions to high-dimensional inverse problems, the efficiency and accuracy of the instrument response representation are of high importance for the imaging process. For efficiency reasons, point spread functions, which make up a large fraction of the response functions of telescopes and microscopes, are usually assumed to be spatially invariant in a given field of view and can thus be represented by a convolution. For many instruments, this assumption does not hold and degrades the accuracy of the instrument representation. Here, we discuss the application of butterfly transforms, which are linear neural network structures whose sizes scale sub-quadratically with the number of data points. Butterfly transforms are efficient by design, since they are inspired by the structure of the Cooley-Tukey fast Fourier transform. In this work, we combine them in several ways into butterfly networks, compare the different architectures with respect to their performance and identify a representation that is suitable for the efficient representation of a synthetic spatially variant point spread function up to a 1% error. Furthermore, we show its application in a short synthetic example.

AIROPA IV: Validating point spread function reconstruction on various science cases

We present an analysis of six independent on-sky datasets taken with the Keck-II/NIRC2 instrument. Using the off-axis point spread function (PSF) reconstruction software AIROPA, we extract stellar astrometry, photometry, and other fitting metrics to characterize the performance of this package. We test the effectiveness of AIROPA to reconstruct the PSF across the field of view in varying atmospheric conditions, number and location of PSF reference stars, stellar crowding, and telescope position angle (PA). We compare the astrometric precision and fitting residuals between a static PSF model and a spatially varying PSF model that incorporates instrumental aberrations and atmospheric turbulence during exposures. Most of the fitting residuals we measure show little to no improvement in the variable-PSF mode over the single-PSF mode. For one of the data sets, we find photometric performance is significantly improved (by ∼10 × ) by measuring the trend seen in photometry as a function of off-axis location. For nearly all other metrics we find comparable astrometric and photometric precision across both PSF modes, with a ∼13 % smaller astrometric uncertainty in variable-PSF mode in the best case. We largely confirm that the spatially variable PSF does not significantly improve the astrometric and other PSF fitting residuals over the static PSF for on-sky observations. We attribute this to unaccounted instrumental aberrations that are not characterized through afternoon adaptive optics (AO) bench calibrations.

Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging.

Single-molecule localization microscopy in a typical wide-field setup has been widely used for investigating subcellular structures with super resolution; however, field-dependent aberrations restrict the field of view (FOV) to only tens of micrometers. Here, we present a deep-learning method for precise localization of spatially variant point emitters (FD-DeepLoc) over a large FOV covering the full chip of a modern sCMOS camera. Using a graphic processing unit-based vectorial point spread function (PSF) fitter, we can fast and accurately model the spatially variant PSF of a high numerical aperture objective in the entire FOV. Combined with deformable mirror-based optimal PSF engineering, we demonstrate high-accuracy three-dimensional single-molecule localization microscopy over a volume of ~180 × 180 × 5 μm3, allowing us to image mitochondria and nuclear pore complexes in entire cells in a single imaging cycle without hardware scanning; a 100-fold increase in throughput compared to the state of the art.

Rethinking data-driven point spread function modeling with a differentiable optical model

In astronomy, upcoming space telescopes with wide-field optical instruments have a spatially varying point spread function (PSF). Specific scientific goals require a high-fidelity estimation of the PSF at target positions where no direct measurement of the PSF is provided. Even though observations of the PSF are available at some positions of the field of view (FOV), they are undersampled, noisy, and integrated into wavelength in the instrument’s passband. PSF modeling represents a challenging ill-posed problem, as it requires building a model from these observations that can infer a super-resolved PSF at any wavelength and position in the FOV. Current data-driven PSF models can tackle spatial variations and super-resolution. However, they are not capable of capturing PSF chromatic variations. Our model, coined WaveDiff, proposes a paradigm shift in the data-driven modeling of the point spread function field of telescopes. We change the data-driven modeling space from the pixels to the wavefront by adding a differentiable optical forward model into the modeling framework. This change allows the transfer of a great deal of complexity from the instrumental response into the forward model. The proposed model relies on efficient automatic differentiation technology and modern stochastic first-order optimization techniques recently developed by the thriving machine-learning community. Our framework paves the way to building powerful, physically motivated models that do not require special calibration data. This paper demonstrates the WaveDiff model in a simplified setting of a space telescope. The proposed framework represents a performance breakthrough with respect to the existing state-of-the-art data-driven approach. The pixel reconstruction errors decrease six-fold at observation resolution and 44-fold for a 3x super-resolution. The ellipticity errors are reduced at least 20 times, and the size error is reduced more than 250 times. By only using noisy broad-band in-focus observations, we successfully capture the PSF chromatic variations due to diffraction. WaveDiff source code and examples associated with this paper are available at this link .

AIROPA II: modeling instrumental aberrations for off-axis point spread functions in adaptive optics

Images obtained with single-conjugate adaptive optics (AO) show spatial variation of the point spread function (PSF) due to both atmospheric anisoplanatism and instrumental aberrations. The poor knowledge of the PSF across the field of view strongly impacts the ability to take full advantage of AO capabilities. The AIROPA project aims to model these PSF variations for the NIRC2 imager at the Keck Observatory. Here, we present the characterization of the instrumental phase aberrations over the entire NIRC2 field of view and we present a new metric for quantifying the quality of the calibration, the fraction of variance unexplained (FVU). We used phase diversity measurements obtained on an artificial light source to characterize the variation of the aberrations across the field of view and their evolution with time. We find that there is a daily variation of the wavefront error (RMS of the residuals is 94~nm) common to the whole detector, but the differential aberrations across the field of view are very stable (RMS of the residuals between different epochs is 59~nm). This means that instrumental calibrations need to be monitored often only at the center of the detector, and the much more time-consuming variations across the field of view can be characterized less frequently (most likely when hardware upgrades happen). Furthermore, we tested AIROPA's instrumental model through real data of the fiber images on the detector. We find that modeling the PSF variations across the field of view improves the FVU metric by 60\% and reduces the detection of fake sources by 70\%.

Galaxy deblending using residual dense neural networks

We present a new neural network approach for deblending galaxy images in astronomical data using Residual Dense Neural network (RDN) architecture. We train the network on synthetic galaxy images similar to the typical arrangements of field galaxies with a finite point spread function (PSF) and realistic noise levels. The main novelty of our approach is the usage of two distinct neural networks: i) a deblending network which isolates a single galaxy postage stamp from the composite and, ii) a classifier network which counts the remaining number of galaxies. The deblending proceeds by iteratively peeling one galaxy at a time from the composite until the image contains no further objects as determined by the classifier, or by other stopping criteria. By looking at the consistency in the outputs of the two networks, we can assess the quality of the deblending. We characterize the flux and shape reconstructions in different quality bins and compare our deblender with the industry standard, SExtractor. We also discuss possible future extensions for the project with variable PSFs and noise levels.