Space-variant Point Spread Function Research Articles

Adaptive anisotropic pixel-by-pixel correction method for a space-variant degraded image.

Large field-of-view optical imaging systems often face challenges in the presence of space-variant degradation. The existence of degradation leads to target detection and recognition being difficult or even unsuccessful. To address this issue, this paper proposes an adaptive anisotropic pixel-by-pixel space-variant correction method. First, we estimated region acquisition of local space-variant point spread functions (PSFs) based on Haar wavelet degradation degree distribution, and obtained initial PSF matrix estimation with inverse distance weighted spatial interpolation. Then, we established a pixel-by-pixel space-variant correction model based on the PSF matrix. Third, we imposed adaptive sparse regularization terms of the Haar wavelet based on the adaptive anisotropic iterative reweight strategy and non-negative regularization terms as the constraint in the pixel-by-pixel space-variant correction model. Finally, as the correction process is refined to each pixel, the split-Bregman multivariate separation solution algorithm was employed for the pixel-by-pixel spare-variant correction model to estimate the final PSF matrix and the gray value of each pixel. Through this algorithm, the "whole image correction" and "block correction" is avoided, the "pixel-by-pixel correction" is realized, and the final corrected images are obtained. Experimental results show that compared with the current advanced correction methods, the proposed approach in the space-variant wide field correction of a degraded image shows better performance in preserving the image details and texture information.

Impact of Windshield Optical Aberrations on Visual Range Camera Based Classification Tasks Performed by CNNs

Cameras operating in the visual range of the electromagnetic spectrum are central to advanced driver assistance systems (ADAS). Front cameras, analyzing traffic, are often located behind the windshield to detect and classify objects.Thus, the area of the windshield within the camera’s field of view is a part of the optical system. Simple windshields consist of two curved glass surfaces connected by a thermoplastic interlayer. Due to defects present in the raw glass, as well as those introduced during the bending and lamination process, windshields will have optical aberrations. While optical quality may be suitable for human vision, it can fall short of what is needed for machine vision. In this article we investigate how the optical aberrations generated by laminated safety glass (LSG) influence the optical performance of a camera system and based on this, how the classification of image content by a convolutional neural network (CNN) is affected. A method for wavefront measurements of LSG samples is presented, which allows us to parameterize a linear optical model in Zernike Space. From this, we derive space-variant point spread functions (PSFs) and apply those to the dataset to simulate the windshield’s impact on the camera image. As a use case, a CNN was trained on the unmodified dataset and compared to the modified versions with the LSG models applied. We measured and modelled two different LSG samples, one with high and the other one with low optical quality. We compare the prediction accuracy of the classification with the unmodified data. The highquality sample had negligible effect on the overall classification accuracy, while the low-quality sample lowered the prediction accuracy by up to ten percentage points due to the optical aberrations.

Correction of aberration-induced phase errors in phase measuring deflectometry.

Phase measuring deflectometry is a powerful measuring method of complex optical surfaces that captures the reflected fringe images associated with a displaying screen and calculates the normal vectors of the surface under test (SUT) accordingly. The captured images are usually set conjugate to the SUT, which in turn makes the screen defocused. As a result, the blurring effect caused by the defocus and aberrations of the off-axis catadioptric imaging system can severely degrade the phases solved from the blurred images. In order to correct the phase errors, the space-variant point spread functions (PSFs) are modeled using a skew-normal function. The phase bias is estimated by forward convolution between the captured images and the PSF models. Demonstrated with a highly curved aspheric surface, the measurement accuracy can be improved by three times.

Adaptive Phase Correction for Phase Measuring Deflectometry Based on Light Field Modulation

The phase measuring deflectometry is a powerful measuring method for complex optical surfaces, which captures the reflected fringe images associated with a displaying screen and calculates the normal vectors of the surface under test (SUT) accordingly. The captured images are usually set conjugate to the SUT, which in turn makes the screen defocused. As a result, the blurring effect caused by the point spread function (PSF) of the off-axis catadioptric imaging system can bias the solved phases. In order to correct the phase errors, a light field is constructed based on the Fourier compressive sensing method to describe the light transmission between the screen and camera pixels. Fringe modulation is conducted to enhance the robustness against noise, and then, space-variant PSFs can be extracted from the light field. The true phases are obtained by solving a Wiener deconvolution problem, with the merit function adaptively regularized by adjusting the damping parameter. The proposed method can correct adaptively the phase errors induced by the complex aberrations. Compared to the reference measurements, the form accuracy can be improved by four times.

PSF Estimation Method of Simple-Lens Camera Using Normal Sinh-Arcsinh Model Based on Noise Image Pairs

With the development of computational photography, single-lens camera combined with corresponding image deblurring algorithm is gradually becoming a new research direction, replacing complex modern optical imaging system such as single lens reflex (SLR) camera. For single-lens camera, the Point Spread Function (PSF) estimation accuracy will directly affect the image restoration effect. In this paper, we designed the simple-lens cameras with one, two and three lenses, respectively, and propose a robust and accurate PSF estimation method of simple-lens camera. The key point of estimation is to obtain the blur image and clear image pairs, which are necessary for non-blind deconvolution PSF estimation. Considering the structure characteristic of simple-lens camera, we take picture of original clear image displayed on the computer screen to get the image pairs through corner detection and color correction is made to remove color distortion. In addition, a few studies have shown that the PSF of the simple lens is close to the spatially deformed wedge, so we use a more reasonable Normal Sinh-Arcsinh (NSAS) model to fit the blur kernel and get its parameters by Powell algorithm. The experiment results have shown that the space-variant PSF estimated by the proposed method achieves better performance than the compared methods both qualitatively and quantitatively.

Deep learning for a space-variant deconvolution in galaxy surveys

The deconvolution of large survey images with millions of galaxies requires developing a new generation of methods that can take a space-variant point spread function into account. These methods have also to be accurate and fast. We investigate how deep learning might be used to perform this task. We employed a U-net deep neural network architecture to learn parameters that were adapted for galaxy image processing in a supervised setting and studied two deconvolution strategies. The first approach is a post-processing of a mere Tikhonov deconvolution with closed-form solution, and the second approach is an iterative deconvolution framework based on the alternating direction method of multipliers (ADMM). Our numerical results based on GREAT3 simulations with realistic galaxy images and point spread functions show that our two approaches outperform standard techniques that are based on convex optimization, whether assessed in galaxy image reconstruction or shape recovery. The approach based on a Tikhonov deconvolution leads to the most accurate results, except for ellipticity errors at high signal-to-noise ratio. The ADMM approach performs slightly better in this case. Considering that the Tikhonov approach is also more computation-time efficient in processing a large number of galaxies, we recommend this approach in this scenario.

Primal–Dual Optimization Strategy With Total Variation Regularization for Prestack Seismic Image Deblurring

Seismic image, especially for the prestack image, performs a blurred version of the reflectivity image due to spatial aliasing, poor acquisition aperture, and nonuniform illumination. The blurring effects can be quantified by the point spread function (PSF). We herein adopt an explicit space-variant PSF formula, which can be defined as a sequential application of the modeling and migration operators with the asymptotic Green's function. The deblurred images are restored using the nonstationary deconvolution with total variation regularization in which the blurred images are described by the convolution between the space-variant PSF and the reflectivity image. However, nonstationary deconvolution is computationally challenging. We introduce an extending primal-dual hybrid gradient (E-PDHG) method to decompose the complex problem into a sequence of simple subproblems that have closed-form solutions. Numerical results on synthetic data and field data demonstrate that the proposed E-PDHG method outperforms the basic PDHG method in the prestack seismic image deblurring.

Space-variant point spread function measurement and interpolation at any depth based on single-pixel imaging.

Point spread function (PSF) is important for evaluating an optical system and image deblurring. In this paper, we proposed a method to measure space-variant PSF at any depth based on single-pixel imaging (SPI), and we initiated a depth-variant PSF interpolation model. In our method, we regarded space-variant PSF as light transport coefficients from object points to image pixels. By applying SPI to each image pixel to obtain these light transport coefficients at different depths, the PSF of each object point can be extracted. The depth calculation of PSF is based on multi-frequency heterodyne phase-shifting principles and perspective-n-point (PnP) algorithm. In our PSF interpolation model, we interpolated the light transport coefficients from different object points to an image pixel first. We then obtained the interpolated PSF indirectly from the interpolated coefficients. With simple experimental facilities containing a digital camera and a liquid crystal display (LCD) screen to display and capture specific patterns, which relative distance is changed, the proposed method accurately obtained the space-variant PSF at any depth. Without complicated calculation, PSF at a certain depth can be interpolated from the PSF measured data at another depth with our PSF interpolation method. Significant similarities exist between the interpolated PSF and directly measured PSF. Our work is a successful attempt in using SPI to solve traditional optical problems.

Applying Space-Variant Point Spread Function to Three-Dimensional Reconstruction of Fluorescence Microscopic Images

Three-dimensional (3D) reconstruction of fluorescence microscopic images is a challenging topic in the image processing, because the imaging system is very complex, and the point spread function (PSF) continuously varies along the optical axis. Generally, the more exact the PSF is, the higher the reconstruction accuracy is. An image reconstruction method is proposed for fluorescence microscopic sample based on space-variant PSF (SV-PSF) which is generated by cubic spline theory in this paper. Firstly, key PSFs are estimated by blind deconvolution algorithm at several depths of fluorescence microscopic image stack along the optical axis. Then, other PSFs are interpolated using cubic spline theory. Finally, a 3D microscopic specimen model is reconstructed by this group of SV-PSFs. The experimental results show that the proposed method is obviously superior to the method in which space-invariant (SI) PSF is used to reconstruct the simulated and real fluorescence microscopic images.

A Space-Variant Deblur Method for Focal-Plane Microwave Imaging

In the research of passive millimetre wave (PMMW) imaging, the focal plane array (FPA) can realize fast, wide-range imaging and detection. However, it has suffered from a limited aperture and off-axis aberration. Thus, the result of FPA is usually blurred by space-variant point spread function (SVPSF) and is hard to restore. In this paper, a polar-coordinate point spread function (PCPSF) model is presented to describe the circle symmetric characteristic of space-variant blur, and a log-polar-coordinate transformation (LPCT) method is propagated as the pre-processing step before the Lucy–Richardson algorithm to eliminate the space variance of blur. Compared with the traditional image deblur method, LPCT solves the problem by analyzing the physical model instead of the approximating it, which has proved to be a feasible way to deblur the FPA imaging system.

Off-axis point spread function characterization in laser guide star adaptive optics systems

Adaptive optics (AO) restore the angular resolution of ground-based telescopes, but at the cost of delivering a time-and space-varying point spread function (PSF) with a complex shape. PSF knowledge is crucial for breaking existing limits on the measured accuracy of photometry and astrometry in science observations. In this paper, we concentrate our analyses on anisoplanatism signature only onto PSF: for large-field observations (20) with single-conjugated AO, PSFs are strongly elongated due to anisoplanatism that manifests itself as three different terms for Laser-guide star (LGS) systems: angular, focal and tilt. We propose a generalized model that relies on a point-wise decomposition of the phase and encompasses the non-stationarity of LGS systems. We demonstrate it is more accurate and less computationally demanding than existing models: it agrees with end-to-end physical-optics simulations to within 0.1% of PSF measurables, such as Strehl-ratio, FWHM and fraction of variance unexplained. Secondly, we study off-axis PSF modelling is with respect to C 2 n (h) profile (heights and fractional weights). For 10 m class telescope, PSF morphology is estimated at 1%-level as long as we model the atmosphere with at least 7 layers whose heights and weights are known respectively with 200m and 10%-precision. As a verification test we used the Canada's NRC-Herzberg HeNOS testbed data, featuring four lasers. We highlight capability of retrieving off-axis PSF characteristics within 10% of fraction of variance unexplained, which complies with the expected range from the sensitivity analysis. Our new off-axis PSF modelling method lays the groundwork for testing on-sky in the near future.

Measure and model a 3-D space-variant PSF for fluorescence microscopy image deblurring.

Conventional deconvolution methods assume that the microscopy system is spatially invariant, introducing considerable errors. We developed a method to more precisely estimate space-variant point-spread functions from sparse measurements. To this end, a space-variant version of deblurring algorithm was developed and combined with a total-variation regularization. Validation with both simulation and real data showed that our PSF model is more accurate than the piecewise-invariant model and the blending model. Comparing with the orthogonal basis decomposition based PSF model, our proposed model also performed with a considerable improvement. We also evaluated the proposed deblurring algorithm. Our new deblurring algorithm showed a significantly better signal-to-noise ratio and higher image quality than those of the conventional space-invariant algorithm.

Lensless photography with only an image sensor.

Photography usually requires optics in conjunction with a recording device (an image sensor). Eliminating the optics could lead to new form factors for cameras. Here, we report a simple demonstration of imaging using a bare CMOS sensor that utilizes computation. The technique relies on the space variant point-spread functions resulting from the interaction of a point source in the field of view with the image sensor. These space-variant point-spread functions are combined with a reconstruction algorithm in order to image simple objects displayed on a discrete LED array as well as on an LCD screen. We extended the approach to video imaging. Finally, we performed experiments to analyze the parametric impact of the object distance. Improving the sensor designs and reconstruction algorithms can lead to useful cameras without optics.

Space variant deconvolution of galaxy survey images

Removing the aberrations introduced by the Point Spread Function (PSF) is a fundamental aspect of astronomical image processing. The presence of noise in observed images makes deconvolution a nontrivial task that necessitates the use of regularisation. This task is particularly difficult when the PSF varies spatially as is the case for the Euclid telescope. New surveys will provide images containing thousand of galaxies and the deconvolution regularisation problem can be considered from a completely new perspective. In fact, one can assume that galaxies belong to a low-rank dimensional space. This work introduces the use of the low-rank matrix approximation as a regularisation prior for galaxy image deconvolution and compares its performance with a standard sparse regularisation technique. This new approach leads to a natural way to handle a space variant PSF. Deconvolution is performed using a Python code that implements a primal-dual splitting algorithm. The data set considered is a sample of 10 000 space-based galaxy images convolved with a known spatially varying Euclid-like PSF and including various levels of Gaussian additive noise. Performance is assessed by examining the deconvolved galaxy image pixels and shapes. The results demonstrate that for small samples of galaxies sparsity performs better in terms of pixel and shape recovery, while for larger samples of galaxies it is possible to obtain more accurate estimates of the galaxy shapes using the low-rank approximation.

PSF Estimation of Space-Variant Ultra-Wide Field of View Imaging Systems

Ultra-wide-field of view (UWFOV) imaging systems are affected by various aberrations, most of which are highly angle-dependent. A description of UWFOV imaging systems, such as microscopy optics, security camera systems and other special space-variant imaging systems, is a difficult task that can be achieved by estimating the Point Spread Function (PSF) of the system. This paper proposes a novel method for modeling the space-variant PSF of an imaging system using the Zernike polynomials wavefront description. The PSF estimation algorithm is based on obtaining field-dependent expansion coefficients of the Zernike polynomials by fitting real image data of the analyzed imaging system using an iterative approach in an initial estimate of the fitting parameters to ensure convergence robustness. The method is promising as an alternative to the standard approach based on Shack–Hartmann interferometry, since the estimate of the aberration coefficients is processed directly in the image plane. This approach is tested on simulated and laboratory-acquired image data that generally show good agreement. The resulting data are compared with the results of other modeling methods. The proposed PSF estimation method provides around 5% accuracy of the optical system model.

Super-Resolution Reconstruction of 3D PET Images Using Two Respiratory-Phase Low-Dose CT Images

Positron emission tomography (PET) images usually suffer from incorrect quantification of the radioactive uptake of small lesions due to low spatial resolution. To improve the spatial resolution, we previously proposed super-resolution (SR) algorithms based on wobble scanning. The proposed algorithms, however, require mechanical motion of the patient bed or a system gantry for wobble scanning. In this paper, we propose a framework for SR reconstruction of 3D PET images based on the use of respiratory motion rather than wobble motion. As in the conventional protocol of PET imaging, gated list-mode PET data are acquired in a free breathing condition. In addition, we acquire two low-dose CT images in a breath-hold manner at exhale and inhale phases, without increasing the radiation burden to a patient. Using the two low-dose CT images, we estimate the 4D motion vector field (MVF) and correspondingly generate a virtual 4D CT image that are matched to the 4D PET image. The 3D CT images have much better spatial resolution than PET images and therefore the corresponding estimated 3D MVFs can be considered reliable for PET SR reconstruction. We then estimate space-variant point spread functions (PSFs) in the imaging field of view using a minimum number of PSFs obtained through Monte-Carlo simulations. Finally, SR reconstruction is performed by incorporating the estimated 3D MVFs and space-variant PSFs. In the SR reconstruction, to avoid the resolution degradation in the evenly spaced parallel-beam rebinning and to reduce the computational time on the graphics processing unit, we introduce a parallel-friendly spanned line of response reconstruction technique based on fan-beam reordering. The proposed framework is evaluated via Monte-Carlo simulations with the digital XCAT phantom and via experiments with several patient datasets including moving lung lesions. Both the simulation and experiment results show that the proposed framework provides much clearer organ boundaries as well as more accurate quantitative lesion information than the conventional methods, with a considerable reduction of computational time.

Constraint matrix factorization for space variant PSFs field restoration

In large-scale spatial surveys, the point spread function (PSF) varies across the instrument field of view (FOV). Local measurements of the PSFs are given by the isolated star images. Yet, these estimates may not be directly usable for post-processing because of the observational noise and potentially the aliasing. Given a set of aliased and noisy star images from a telescope, we aim to estimate well-resolved and noise-free PSFs at the observed star positions. In particular, exploiting the spatial correlation of the PSFs across the FOV. We introduce RCA (resolved components analysis) which is a noise-robust dimension reduction and super-resolution method based on matrix-factorization. We propose an original way of using the PSFs spatial correlation in the restoration process through sparsity. The introduced formalism can be applied to correlated data sets with respect to any euclidean parametric space. We tested our method on simulated monochromatic PSFs of Euclid telescope (launch planned for 2020). The proposed method outperforms existing PSFs restoration and dimension reduction methods. We show that a coupled sparsity constraint on individual PSFs and their spatial distribution yields a significant improvement on both the restored PSFs shapes and the PSFs subspace identification, in presence of aliasing. RCA can be naturally extended to account for the wavelength dependency of the PSFs.

A computational photography algorithm for quality enhancement of single lens imaging deblurring

Modern single lens reflex lenses typically consist of up to two dozen individual optical elements, the complexity of which is necessary to compensate for geometric and chromatic aberrations. This paper adopts one single lens to capture images instead of complex lenses, and computational photography technique is employed to remove corresponding imaging artifacts. We initially estimate the space-variant point spread function of the single lens by combining l1/l2 image and sparse kernel priors. A fast non-blind deconvolution method with hyper-Laplacian prior is then performed to recover clear image. Experimental results show that the proposed method is at par with state-of-the-art non-blind deconvolution approaches, especially with regards its speed that is much faster than that of existing methods.

Cannula-based computational fluorescence microscopy

We converted a solid-glass cannula into a high-resolution widefield fluorescence microscope. Calibrating the space-variant point-spread functions of the cannula and applying a nonlinear optimization algorithm to reconstruct object details enable this development. The resolution of our system is ∼1 μm, and fluorophore position is determined to a precision of ∼20 nm. Images of microglia from fixed slices of mouse brains at various post-natal development stages were also obtained.

A method for space-variant deblurring with application to adaptive optics imaging in astronomy

Images from adaptive optics systems are generally affected by significant distortions of the point spread function (PSF) across the field of view, depending on the position of natural and artificial guide stars. Image reduction techniques circumventing or mitigating these effects are important tools to take full advantage of the scientific information encoded in AO images. The aim of this paper is to propose a method for the deblurring of the astronomical image, given a set of samples of the space-variant PSF. The method is based on a partitioning of the image domain into regions of isoplanatism and on applying suitable deconvolution methods with boundary effects correction to each region. The effectiveness of the boundary effects correction is proved. Moreover, the criterion for extending the disjoint sections to partially overlapping sections is validated. The method is applied to simulated images of a stellar system characterized by a spatially variable PSF. We obtain good photometric quality, and therefore good science quality, by performing aperture photometry on the deblurred images. The proposed method is implemented in IDL in the Software Package "Patch", which is available on http://www.airyproject.eu.