Point Spread Function Estimation Research Articles

A novel variational approach for multiphoton microscopy image restoration: from PSF estimation to 3D deconvolution

In multi-photon microscopy (MPM), a recent in-vivo fluorescence microscopy system, the task of image restoration can be decomposed into two interlinked inverse problems: firstly, the characterization of the point spread function (PSF) and subsequently, the deconvolution (i.e. deblurring) to remove the PSF effect, and reduce noise. The acquired MPM image quality is critically affected by PSF blurring and intense noise. The PSF in MPM is highly spread in 3D and is not well characterized, presenting high variability with respect to the observed objects. This makes the restoration of MPM images challenging. Common PSF estimation methods in fluorescence microscopy, including MPM, involve capturing images of sub-resolution beads, followed by quantifying the resulting ellipsoidal 3D spot. In this work, we revisit this approach, coping with its inherent limitations in terms of accuracy and practicality. We estimate the PSF from the observation of relatively large beads (approximately 1 μm in diameter). This goes through the formulation and resolution of an original non-convex minimization problem, for which we propose a proximal alternating method along with convergence guarantees. Following the PSF estimation step, we then introduce an innovative strategy to deal with the high level multiplicative noise degrading the acquisitions. We rely on a heteroscedastic noise model for which we estimate the parameters. We then solve a constrained optimization problem to restore the image, accounting for the estimated PSF and noise, while allowing a minimal hyper-parameter tuning. Theoretical guarantees are given for the restoration algorithm. These algorithmic contributions lead to an end-to-end pipeline for 3D image restoration in MPM, that we share as a publicly available Python software. We demonstrate its effectiveness through several experiments on both simulated and real data.

A semi-blind deconvolution method to estimate and remove optical aberrations

Optical aberrations (ABRs) always hinder high-quality imaging, and designers make a vast effort to correct them. We propose a computational method to estimate and remove an ABR that could blur an image. It works upon the combination of an elementary blind deconvolution process and finding the best correlation between the estimated point spread function (PSF) and a pre-defined basis set of ABR-related PSFs. The process is helpful to approximately extract a non-uniform PSF of an ABR and a sharp image from a single blurry one. We successfully tested the method for a dominant effect of defocusing, spherical, coma, astigmatism, or field curvature in different pictures. All the MATLAB codes are available in the Supplementary Material.

Improvement of Whole-body Bone Planar Images on a Bone-dedicated Single-photon Emission Computed Tomography Scanner by Blind Deconvolution Algorithm

Purpose: We have developed a bone-dedicated collimator with higher sensitivity but slightly degraded resolution on single-photon emission computed tomography (SPECT) for planar bone scintigraphy, compared with conventional low-energy high-resolution collimator. In this work, we investigated the feasibility of using the blind deconvolution algorithm to improve the resolution of planar images on bone scintigraphy. Materials and Methods: Monte Carlo simulation was performed with the NCAT phantom for modeling bone scintigraphy on the clinical dual-head SPECT scanner (Imagine NET 632, Beijing Novel Medical Equipment Ltd.) equipped with the bone-dedicated collimator. Maximum likelihood estimation method was used for the blind deconvolution algorithm. The initial estimation of point spread function (PSF) and iteration number for the method were determined by comparing the deblurred images obtained from different input parameters. We simulated different tumors in five different locations and with five different diameters to evaluate the robustness of the initial inputs. Furthermore, we performed chest phantom studies on the clinical SPECT scanner. The quantified increased contrast ratio (CR) between the tumor and the background was evaluated. Results: The 2 mm PSF kernel and 10 iterations provided a practical and robust deblurred image on our system. Those two inputs can generate robust deblurred images in terms of the tumor location and size with an average increased CR of 21.6%. The phantom studies also demonstrated the ability of blind deconvolution, using those two inputs, with increased CRs of 17%, 17%, 22%, 20%, and 13% for lesions with diameters of 1 cm, 2 cm, 3 cm, 4 cm, and 5 cm, respectively. Conclusions: It is feasible to use the blind deconvolution algorithm to deblur the planar images for SPECT bone scintigraphy. The appropriate values of the PSF kernel and the iteration number for the blind deconvolution can be determined using simulation studies.

Multi-shaping sparse-continuous reconstruction for an optical coherence tomography sidelobe suppression.

Optical coherence tomography (OCT) images are commonly affected by sidelobe artifacts due to spectral non-uniformity and spectral leakage. Conventional frequency domain spectral shaping methods widen the mainlobe and compromise axial resolution. While image-domain deconvolution techniques can address the trade-off between axial resolution and artifact suppression, their reconstruction quality relies on accurate measurement or estimation of system point spread function (PSF). Inaccurate PSF estimation leads to loss of details in the reconstructed images. In this Letter, we introduce multi-shaping sparse-continuous reconstruction (MSSCR) for an OCT image, a novel, to the best of our knowledge, framework that combines spectral multi-shaping and iterative image reconstruction with sparse-continuous priors. The MSSCR achieves sidelobe suppression without requiring any PSF measurement or estimation and effectively preserving the axial resolution. The experimental results demonstrate that the MSSCR achieves sidelobe suppression of more than 8 dB. We believe that the MSSCR holds potential for addressing sidelobe artifacts in OCT.

Development of Adaptive Point-Spread Function Estimation Method in Various Scintillation Detector Thickness for X-ray Imaging.

An indirect conversion X-ray detector uses a scintillator that utilizes the proportionality of the intensity of incident radiation to the amount of visible light emitted. A thicker scintillator reduces the patient's dose while decreasing the sharpness. A thin scintillator has an advantage in terms of sharpness; however, its noise component increases. Thus, the proposed method converts the spatial resolution of radiographic images acquired from a normal-thickness scintillation detector into a thin-thickness scintillation detector. Note that noise amplification and artifacts were minimized as much as possible after non-blind deconvolution. To accomplish this, the proposed algorithm estimates the optimal point-spread function (PSF) when the structural similarity index (SSIM) and feature similarity index (FSIM) are the most similar between thick and thin scintillator images. Simulation and experimental results demonstrate the viability of the proposed method. Moreover, the deconvolution images obtained using the proposed scheme show an effective image restoration method in terms of the human visible system compared to that of the traditional PSF measurement technique. Consequently, the proposed method is useful for restoring degraded images using the adaptive PSF while preventing noise amplification and artifacts and is effective in improving the image quality in the present X-ray imaging system.

Magnetic particle imaging deblurring with dual contrastive learning and adversarial framework

Magnetic particle imaging (MPI) is an emerging medical imaging technique that has high sensitivity, contrast, and excellent depth penetration. In MPI, x-space is a reconstruction method that transforms the measured voltages into particle concentrations. The reconstructed native image can be modeled as a convolution of the magnetic particle concentration with a point-spread function (PSF). The PSF is one of the important parameters in deconvolution. However, accurately measuring or modeling the PSF in the hardware used for deconvolution is challenging due to the various environment and magnetic particle relaxation. The inaccurate PSF estimation may lead to the loss of the content structure of the MPI image, especially in low gradient fields. In this study, we developed a Dual Adversarial Network (DAN) with patch-wise contrastive constraint to deblur the MPI image. This method can overcome the limitations of unpaired data in data acquisition scenarios and remove the blur around the boundary more effectively than the common deconvolution method. We evaluated the performance of the proposed DAN model on simulated and real data. Experimental results confirmed that our model performs favorably against the deconvolution method that is mainly used for deblurring the MPI image and other GAN-based deep learning models.

A motion-blurred restoration method for surface damage detection of wind turbine blades

In order to reduce expensive cost and downtimes, visual inspection method based on unmanned aerial vehicles has become a promising method for surface damage detection in wind turbine blades. Due to the fact that the linear velocity of points on the blade varies along its longitudinal dimension while the blade is in-operation, the captured blade image suffers degradation caused by motion-blur, and the degradation process of the image is nonlinear. To address this issue, an area-based deblurring method is firstly developed. The points on each divided region can be assumed to follow a uniform motion with the same linear velocity and blurring effect, which indicates that the nonlinear problem can be resolved by linear methods. Point spread function estimation is then conducted to each region to obtain the blur kernels, and image restoration of each region is carried out by using Wiener filter algorithm. Ultimately, image fusion is implemented to merge all the restored regions to realize blade image restoration. Experimental results and evaluation have demonstrated that the proposed area-based method can mitigate blurring effects of rotating blade images, thus further improve the detection accuracy for the on-line inspection.

Veracious Interpolated Measure of Angle and Length for Underwater Motion Blurred Images

The quality of the underwater image is impairing with the atmospheric conditions. In this, one of the most significant issues in recent days is due to the motion blur induced by the imaging device or by the movement of the object in underwater image quality degradation. The various parameters of the blurred image must be identified to fix the effect of blurring in post-imaging. Therefore, spectrum-based parameter estimation method is proposed. Initially, to estimate the point spread function (PSF), the angle and the length is measured from image spectrum using radon transform. Then, for the accurate estimation of PSF, Optimized Polynomial Lagrange Interpolation (OPLI) is proposed. The data were collected and analyzed in various natural and structured water bodies in Chennai without affecting the real environment. It is observed that for the underwater images collected, the proposed OPLI approach outperforms compared to few existing traditional estimation methods like cepstral, hough, and radon. Then this veracious interpolated measure of angle and length (VIMAL) is restored using modified Lucy algorithm and is evaluated which results in high performance than the existing classical state-of-the-art methods.

Noninvasive Imaging Through Scattering Media with Enlarged FOV Using PSF Estimations and Correlations

Noninvasive imaging through scattering media using memory effect (ME) normally suffers from the failure of imaging when the object is beyond the ME region, limiting its field of view (FOV). Herein, a method to enlarge the FOV is proposed by utilizing point‐spread‐function (PSF) estimations and correlations. For the first time, to the best of the knowledge, it is successfully demonstrated that a whole object or multiple objects with the right relative positions and orientations, although they are beyond the ME region, can be noninvasively well reconstructed using the proposed method, which suggests that it can realize noninvasive imaging through scattering media with enlarged FOV. Further, it is experimentally verified that the method can promote one of the previous works, making it capable of noninvasive super‐resolution imaging through scattering media beyond the ME region. The technique paves the way to noninvasively visualize objects behind scattering media beyond the ME region with enlarged FOV and unprecedented clarity.

Image-to-image translation for wavefront and point spread function estimation

We develop and evaluate a new approach to phase estimation for observational astronomy that can be used for accurate point spread function reconstruction. Phase estimation is required where a terrestrial observatory uses an adaptive optics (AO) system to assist astronomers in acquiring sharp, high-contrast images of faint and distant objects. Our approach is to train a conditional adversarial artificial neural network architecture to predict phase using the wavefront sensor data from a closed-loop AO system. We present a detailed simulation study under different turbulent conditions, using the retrieved residual phase to obtain the point spread function of the simulated instrument. Compared to the state-of-the-art model-based approach in astronomy, our approach is not explicitly limited by modeling assumptions, e.g., independence between terms, such as bandwidth and anisoplanatism—and is conceptually simple and flexible. We use the open-source COMPASS tool for end-to-end simulations. On key quality metrics, specifically the Strehl ratio and Halo distribution in our application domain, our approach achieves results better than the model-based baseline.

Improving axial resolution based on the deconvolution recovery method combined with adaptive weighting techniques for ultrasound imaging.

A fundamental challenge in medical ultrasound imaging is to improve the resolution accurately. Adaptive beamforming is often used to improve lateral resolution, such as minimum variance (MV) and phase coherence factor (PCF). However, it is difficult to improve the axial resolution due to the limitation of the spatial pulse length (SPL) of the transmitted signal. A deconvolution recovery method combines two adaptive weighting techniques to improve axial resolution. A deconvolution recovery (DR) technique is used to improve axial resolution with a shorter SPL. Then, the DR is combined with MV and PCF (DR-MVPCF) to suppress the sidelobe. The influence of different transmission modes, regularization parameters, and the estimation of point spread function are discussed on the proposed algorithm. In simulation, DR-MVPCF improved axial resolution from 0.41 mm (0.98 λ) to 0.09 mm (0.21 λ) compared with MV-PCF. In the water bath experiment, DR-MVPCF provided improvement of axial resolution from 0.39 mm (0.93 λ) to 0.07 mm (0.17 λ) compared with MV-PCF. In-vivo data experiment, the DR-MVPCF method increased the speckle signal-to-noise ratio and visibility of the structure while the contrast ratio and contrast-noise ratio decreased. The proposed method can improve the axial resolution significantly.

Improvement of Point Spread Function Estimation Method for Motion Blurred Image and Image Restoration

The estimation of point spread function (PSF) of motion-blurred images is the key to image restoration, which is mainly affected by two physical quantities: blur angle and blur length. The traditional point spread function estimation method is easy to be disturbed by the bright cross in the centre of the blurred image spectrum when estimating the fuzzy angle, which affects the quality of image restoration. In addition, when using the traditional Radon transform to detect fuzzy angles, the area of bright spots in the picture is large, which will cause large fluctuations in the detection results. Therefore, this paper proposes a cepstrum analysis method for motion blurred images. Firstly, the gradient cepstrum estimation of the blurred image is given, and the cepstrum is binarized and further refined by setting the gray threshold of the cepstrum. Finally, the line fitting is performed on the refined binary image, and the slope of the fitting line is calculated, so that the point spread function of the blurred image can be accurately and directly estimated, and better image restoration effect can be obtained.

Analysis of influence of imaging resolution of relay lens on image reconstruction quality in pixel-wise coded exposure imaging technology

Pixel-wise coded exposure (PCE) imaging based on digital micromirror device (DMD) is an advanced high-speed imaging technology, which can realize the high-speed imaging by using a low-frame-rate camera. During exposure time, the multi-frame image information of a dynamic object can be integrated into one encoded image, and then the multi-frame sub-exposure images can be extracted by the post-processing algorithm. Therefore, the accurate pixel-to-pixel alignment between the DMD and the camera is the key step to realize PCE imaging, which has drawn much attention from researchers. So their studies mainly focused on how to achieve accurate pixel matching. However, the resolution of the relay imaging lens, as another important influence factor of PCE imaging, also has a significant influence on the imaging results, but few people have studied and analyzed it. To solve this problem, in this work, we theoretically analyze the influence of the resolution of the relay imaging system on the reconstructed decoded images, and verifies the theoretical analysis through simulation and imaging experiments. On this basis, a PCE imaging system is built, and a point spread function (PSF) estimation method of relay lens based on the fringe phase is proposed. Furthermore, a Richard-Lucy deconvolution algorithm is introduced into the reconstruction process of coded image to effectively improve the quality of PCE imaging, which is of great significance in developing the PCE imaging technology.

Point Spread Function Calculation for Unfocused Plenoptic Camera by Random Forest

A novel calculation scheme is proposed for the point spread function (PSF) of the unfocused plenoptic camera, which describes the response of image patterns to the point light source locations. It is based on the data-driven machine learning method, random forest (RF). Both synthetic and experimental images are obtained to build the datasets for training and test. By feature construction, transformed variables are selected as the inputs, the intensity values of the illuminated pixels are predicted by the RF. PSF estimation errors of the current scheme are compared with that of the widely used geometrical ray tracing method. The results demonstrate that the method by RF is obviously more accurate and robust, especially for the regions close to the focal plane, mitigating the estimation errors induced by optical aberrations.

Three-dimensional point spread function estimation method for mid-wave infrared microscope imaging.

A three-dimensional point spread function experimental estimation method based on the system's focal plane array spatial local impulse response of a mid-wave infrared microscope is presented. The method uses several out-of-focus two-dimensional point spread function planes to achieve a single three-dimensional point spread function of the whole microscope's optical spreading, expanding the limits of infrared optical technology by one dimension. This technique includes stages of image acquisition, nonuniformity correction, filtering, and multi-planar reconstruction steps, and its effectiveness is demonstrated on biological sample image restoration by means of a multi-planar refocusing application.

Scatter imaging with super-memory effect based on adaptive β-parameter hybrid input-output algorithm-assisted estimation of point spread function

It is of great significance to realize the imaging of the target behind the scattering medium, which is often limited by the optical memory effect range. In this paper, a non-invasive super-memory effect scattering imaging method based on adaptive β-parameter hybrid input-output (APHIO) algorithm-assisted point spread function (PSF) estimation is proposed. An iterative algorithm APHIO is designed to use the speckle autocorrelation region to constrain the iterative region and optimize the value of the parameter β in the traditional HIO algorithm. Besides, the estimated PSF of the imaging system is combined with a new scanning method to better reconstruct the target object with super-memory effect range. The results show that the APHIO algorithm has the characteristics of faster convergence and more stable reconstruction quality than the traditional HIO algorithm. The overall imaging framework has a good performance in the reconstruction efficiency of the target with super-memory effect range.

Adaptive sparse reconstruction for lensless digital holography via PSF estimation and phase retrieval.

In-line lensless digital holography has great potential in multiple applications; however, reconstructing high-quality images from a single recorded hologram is challenging due to the loss of phase information. Typical reconstruction methods are based on solving a regularized inverse problem and work well under suitable image priors, but they are extremely sensitive to mismatches between the forward model and the actual imaging system. This paper aims to improve the robustness of such algorithms by introducing the adaptive sparse reconstruction method, ASR, which learns a properly constrained point spread function (PSF) directly from data, as opposed to solely relying on physics-based approximations of it. ASR jointly performs holographic reconstruction, PSF estimation, and phase retrieval in an unsupervised way by maximizing the sparsity of the reconstructed images. Like traditional methods, ASR uses the image formation model along with a sparsity prior, which, unlike recent deep learning approaches, allows for unsupervised reconstruction with as little as one sample. Experimental results in synthetic and real data show the advantages of ASR over traditional reconstruction methods, especially in cases where the theoretical PSF does not match that of the actual system.

Companion mass limits for 17 binary systems obtained with binary differential imaging and MagAO/Clio

ABSTRACT Improving direct detection capability close to the star through improved star subtraction and post-processing techniques is vital for discovering new low-mass companions and characterizing known ones at longer wavelengths. We present results of 17 binary star systems observed with the Magellan adaptive optics system (MagAO) and the Clio infrared camera on the Magellan Clay Telescope using binary differential imaging (BDI). BDI is an application of reference differential imaging (RDI) and angular differential imaging (ADI) applied to wide binary star systems (2 arcsec <Δρ < 10 arcsec) within the isoplanatic patch in the infrared. Each star serves as the point spread function (PSF) reference for the other, and we performed PSF estimation and subtraction using principal component analysis. We report contrast and mass limits for the 35 stars in our initial survey using BDI with MagAO/Clio in L′ and 3.95 µm bands. Our achieved contrasts varied between systems, and spanned a range of contrasts from 3.0 to 7.5 magnitudes and a range of separations from 0.2 to 2 arcsec. Stars in our survey span a range of masses, and our achieved contrasts correspond to late-type M-dwarf masses down to ∼10 MJup. We also report detection of a candidate companion signal at 0.2 arcsec (18 au) around HIP 67506 A (SpT G5V, mass ∼1.2 M⊙), which we estimate to be $\sim 60-90 \, \rm{M_{Jup}}$. We found that the effectiveness of BDI is highest for approximately equal brightness binaries in high-Strehl conditions.

Incoherent Point Spread Function Estimation and Multipoint Deconvolution for Active Incoherent Millimeter-Wave Imaging

We present an approach to image deconvolution in active incoherent millimeter-wave (AIM) imaging. While traditional incoherent imaging systems capture thermal radiation from the scene, AIM imaging uses the transmission of noise signals to increase the signal-to-noise ratio (SNR) while maintaining the necessary space–time incoherence required for image reconstruction. Images formed by any imaging technique are corrupted by the impulse response, or point spread function (PSF), of the imaging system; however, the degradation of the image due to the PSF can be alleviated significantly via deconvolution if the PSF is characterized. Typically, estimation of the PSF is challenging due to spatial undersampling and the impact of noise. In this work, we experimentally demonstrate an approach to estimate the PSF of AIM imaging systems by averaging PSF estimates from multiple point targets using a shift-and-add method in a 38-GHz imaging system. We demonstrate image deconvolution using the CLEAN algorithm to reconstruct a scene with multiple targets.

Dynamic PSF-based jitter compensation and quality improvement for push-broom optical images considering terrain relief and the TDI effect.

Platform attitude jitter is inevitable during a satellite flight. The jitter reduces the quality of push-broom optical images, resulting in geometric deformation and image blur. This paper proposes an approach that offers jitter effect compensation and quality improvement for push-broom optical images based on dynamic point spread function (PSF) estimation and iterative image restoration considering the effect of terrain relief and time delay integration (TDI). First, the attitude jitter is precisely detected and estimated by considering the terrain relief. The jitter information is then used to establish the dynamic PSF of each image line considering the changing jitter values and TDI effect. Finally, based on the constructed PSF, the image is improved by iterative restoration using the optimal-window Richardson-Lucy algorithm. The method is validated by both simulation and real data experiments. In the simulation experiment, we apply jitter with different amplitudes and frequencies to generate the degraded images and then restore the image using the proposed restoration method. The results show that the proposed method can effectively restore images affected by jitter. In addition, real data experiments are carried out with multispectral remote sensing images from the ZY-3 satellite. The results show that in addition to the improvement of the radiometric quality, the geometric quality is also significantly improved in both the across-track and the along-track directions. The experimental results validated that the proposed method outperformed other methods without considering the terrain and TDI effect.