Photon-limited Images Research Articles

Scanning-based photon-limited imaging through scattering media

Imaging through scattering media has attracted considerable research interest in recent years. Most existing studies are dedicated to reconstructing objects from the detected speckle pattern in an array-based system. However, the inevitable aliasing effect among pixels degrades the imaging quality significantly. Therefore, most existing studies usually could only work well under a weak scattering medium and a short working distance. To address this issue, we innovatively propose a robust scanning-based photon-limited imaging system for imaging through scattering media. To boost the imaging capability, we specifically adopt a point-to-point scanning strategy to eliminate the aliasing effect, and utilize a Geiger-mode avalanche photo diode to detect extremely weak signals after passing through the scattering media. As the images captured by the proposed imaging system always suffer from Poisson noise at unknown levels, we further construct a blind deep-learning based denoising framework with “one-to-all” strategy. By introducing a noise estimation sub-network with dense connection and batch normalization operations into the whole denoising framework, one trained model with synthetic datasets based on the realistic noise model could be generalized well to all real captured photon-limited images. Extensive experiments in real situations demonstrate that significant improvements can be achieved in the proposed method. For example, clear objects can be observed through several thick scattering media under an approximate working distance of 7 m, even with a low-power (average power of approximately 20 mW) light source.

Fast correlated-photon imaging enhanced by deep learning

Quantum imaging using photon pairs with strong quantum correlations has been harnessed to bring quantum advantages to various fields from biological imaging to range finding. Such inherent non-classical properties support the extraction of more valid signals to build photon-limited images, even in low-light conditions where the shot noise becomes dominant as light decreases to a single-photon level. Numerical optimization algorithms are possible but require thousands of photon-sparse frames, and they are thus unavailable in real time. We demonstrate fast correlated-photon imaging enhanced by deep learning as an intelligent computational strategy to discover a deeper structure in big data. Our work verifies that a convolutional neural network can efficiently solve inverse imaging problems associated with strong shot noise and background noise (electronic noise, scattered light). Our results show that we can overcome limitations due to the trade-off between imaging speed and image quality by pushing the low-light imaging technique to the single-photon level in real time, which enables deep-learning-enhanced quantum imaging for real-life applications.

HCLR: A hybrid clustering and low-rank regularization-based method for photon-limited image restoration

A photon-limited image can be represented as a pixel matrix limited by the relatively small number of collected photons. The image can also be seen as being contaminated by Poisson noise because the total number of photons follows the Poisson distribution. Through exploitation of the inherent properties of observation combined with application of a denoising method, an image can be significantly restored. In this paper, a hybrid clustering and low-rank regularization-based model (HCLR) is proposed based on the essential features of patch clustering and noise. An efficient Newton-type method is designed to optimize this biconvex problem. Experimental results demonstrate that HCLR achieves competitive denoising performance, especially for high noise levels, compared with state-of-the-art Poisson denoising algorithms.

Tomographic Image Reconstruction with a Spatially Varying Gamma Mixture Prior

A spatially varying Gamma mixture model prior is employed for tomographic image reconstruction, ensuring effective noise elimination and the preservation of region boundaries. We define a line process, modeling edges between image segments, through appropriate Markov random field smoothness terms which are based on the Student’s t-distribution. The proposed algorithm consists of two alternating steps. In the first step, the mixture model parameters are automatically estimated from the image. In the second step, the reconstructed image is estimated by optimizing the maximum-a-posteriori criterion using the one-step-late expectation–maximization and preconditioned conjugate gradient algorithms. Numerical experiments on various photon-limited image scenarios show that the proposed model outperforms the compared state-of-the-art reconstruction models.

Poisson noise removal based on nonlocal total variation with Euler’s elastica pre-processing

An enhancement-based Poisson denoising method for photon-limited images is presented. The noisy image is firstly pre-processed for enhancing incomplete object information, and then it is denoised while preserving the restored structural details. A variational regularization model based on Euler’s elastica (EE) is proposed for image enhancement pre-processing. A nonlocal total variation (NLTV) regularization model is then employed in the second stage of image denoising. The above two optimization problems are solved by the alternating direction method of multipliers (ADMM). For Poissonian images with low image peak values, experiments demonstrate the validity and efficiency of the proposed method for both restoring geometric structure and removing noise.

A Bayesian Approach to Denoising of Single-Photon Binary Images

This paper discusses new methods for processing images in the photon-limited regime where the number of photons per pixel is binary. We present a new Bayesian denoising method for binary, single-photon images. Each pixel measurement is assumed to follow a Bernoulli distribution whose mean is related by a nonlinear function to the underlying intensity value to be recovered. Adopting a Bayesian approach, we assign the unknown intensity field a smoothness promoting spatial and potentially temporal prior while enforcing the positivity of the intensity. A stochastic simulation method is then used to sample the resulting joint posterior distribution and estimate the unknown intensity, as well as the regularization parameters. We show that this new unsupervised denoising method can also be used to analyze images corrupted by Poisson noise. The proposed algorithm is compared to state-of-the art denoising techniques dedicated to photon-limited images using synthetic and real single-photon measurements. The results presented illustrate the potential benefits of the proposed methodology for photon-limited imaging, in particular with non photon-number resolving detectors.

Integrated circuit authentication using photon-limited x-ray microscopy.

A counterfeit integrated circuit (IC) may contain subtle changes to its circuit configuration. These changes may be observed when imaged using an x-ray; however, the energy from the x-ray can potentially damage the IC. We have investigated a technique to authenticate ICs under photon-limited x-ray imaging. We modeled an x-ray image with lower energy by generating a photon-limited image from a real x-ray image using a weighted photon-counting method. We performed feature extraction on the image using the speeded-up robust features (SURF) algorithm. We then authenticated the IC by comparing the SURF features to a database of SURF features from authentic and counterfeit ICs. Our experimental results with real and counterfeit ICs using an x-ray microscope demonstrate that we can correctly authenticate an IC image captured using orders of magnitude lower energy x-rays. To the best of our knowledge, this Letter is the first one on using a photon-counting x-ray imaging model and relevant algorithms to authenticate ICs to prevent potential damage.

ASSESSMENT OF RESTORATION METHODS OF X-RAY IMAGES WITH EMPHASIS ON MEDICAL PHOTOGRAMMETRIC USAGE

Abstract. Nowadays, various medical X-ray imaging methods such as digital radiography, computed tomography and fluoroscopy are used as important tools in diagnostic and operative processes especially in the computer and robotic assisted surgeries. The procedures of extracting information from these images require appropriate deblurring and denoising processes on the pre- and intra-operative images in order to obtain more accurate information. This issue becomes more considerable when the X-ray images are planned to be employed in the photogrammetric processes for 3D reconstruction from multi-view X-ray images since, accurate data should be extracted from images for 3D modelling and the quality of X-ray images affects directly on the results of the algorithms. For restoration of X-ray images, it is essential to consider the nature and characteristics of these kinds of images. X-ray images exhibit severe quantum noise due to limited X-ray photons involved. The assumptions of Gaussian modelling are not appropriate for photon-limited images such as X-ray images, because of the nature of signal-dependant quantum noise. These images are generally modelled by Poisson distribution which is the most common model for low-intensity imaging. In this paper, existing methods are evaluated. For this purpose, after demonstrating the properties of medical X-ray images, the more efficient and recommended methods for restoration of X-ray images would be described and assessed. After explaining these approaches, they are implemented on samples from different kinds of X-ray images. By considering the results, it is concluded that using PURE-LET, provides more effective and efficient denoising than other examined methods in this research.

ASSESSMENT OF RESTORATION METHODS OF X-RAY IMAGES WITH EMPHASIS ON MEDICAL PHOTOGRAMMETRIC USAGE

Nowadays, various medical X-ray imaging methods such as digital radiography, computed tomography and fluoroscopy are used as important tools in diagnostic and operative processes especially in the computer and robotic assisted surgeries. The procedures of extracting information from these images require appropriate deblurring and denoising processes on the pre- and intra-operative images in order to obtain more accurate information. This issue becomes more considerable when the X-ray images are planned to be employed in the photogrammetric processes for 3D reconstruction from multi-view X-ray images since, accurate data should be extracted from images for 3D modelling and the quality of X-ray images affects directly on the results of the algorithms. For restoration of X-ray images, it is essential to consider the nature and characteristics of these kinds of images. X-ray images exhibit severe quantum noise due to limited X-ray photons involved. The assumptions of Gaussian modelling are not appropriate for photon-limited images such as X-ray images, because of the nature of signal-dependant quantum noise. These images are generally modelled by Poisson distribution which is the most common model for low-intensity imaging. In this paper, existing methods are evaluated. For this purpose, after demonstrating the properties of medical X-ray images, the more efficient and recommended methods for restoration of X-ray images would be described and assessed. After explaining these approaches, they are implemented on samples from different kinds of X-ray images. By considering the results, it is concluded that using PURE-LET, provides more effective and efficient denoising than other examined methods in this research.

Image retrodiction at low light levels

Imaging technologies working at very low light levels acquire data by counting the number of photons impinging on each pixel. Especially in cases with, on average, less than one photocount per pixel, the resulting images are heavily corrupted by Poissonian noise. To tackle this problem, we use methods from Bayesian statistics to retrodict the spatial intensity distribution responsible for the photocount measurements. Unlike the usual photon-limited image denoising algorithms, we calculate the full probability distributions for the intensities at each pixel. The knowledge of these probability distributions helps to assess the validity of results from image analysis using data corrupted by Poisson noise with low photon-count numbers and dark counts.

Photon counting imaging and phase mask multiplexing for multiple images authentication and digital hologram security

We propose an image encryption system based on phase mask multiplexing and photon counting imaging (PCI) technique for multiple image authentications and digital hologram security. Multiple images to be authenticated are converted into phase-only images using a phase-retrieval algorithm in fractional Fourier transform domain. The multiple phase-only images are multiplexed into a single phase-only mask and it is further encrypted into a complex-valued function. The photon-limited image is generated by applying the PCI technique to a complex-valued function. For decryption, the photon-limited decrypted images are obtained after applying the appropriate keys. The photon-limited decrypted image contains sufficient information for verification because this image has sparse representation. To verify the photon-limited decrypted image, the optical correlation filters can be used. The proposed system has also been used for hologram security. For hologram security, the system provides multiple layers of security by hiding multiple encrypted images. The computer simulation results have been presented. The authentication scheme has been tested for securing three-dimensional information through experimentally recorded Fresnel digital hologram.

Photon counting imaging and polarized light encoding for secure image verification and hologam watermarking

We propose an optical image security scheme based on polarized light encoding and the photon counting technique. An input image is encoded using the concept of polarized light, which is parameterized using Stokes–Mueller formalism. The encoded image is further encrypted by applying the photon counting imaging technique to obtain a photon limited image. For decryption, the photon limited decrypted image is obtained by using a polarized light decoding scheme with the help of appropriate keys. The decrypted image has sparse representation, which contains sufficient information for verification. This photon counted decrypted image can be verified using correlation filters. The proposed encryption technique offers benefits over the double random phase encoding in that it does not require active elements such as a lens and provides flexibility in the design of encryption keys. The proposed encryption scheme has also been used for hologram watermarking. The computer simulation results for secure image verification and the hologram watermarking scheme have been presented.

Poisson Noise Reduction with Non-local PCA

Photon-limited imaging arises when the number of photons collected by a sensor array is small relative to the number of detector elements. Photon limitations are an important concern for many applications such as spectral imaging, night vision, nuclear medicine, and astronomy. Typically a Poisson distribution is used to model these observations, and the inherent heteroscedasticity of the data combined with standard noise removal methods yields significant artifacts. This paper introduces a novel denoising algorithm for photon-limited images which combines elements of dictionary learning and sparse patch-based representations of images. The method employs both an adaptation of Principal Component Analysis (PCA) for Poisson noise and recently developed sparsity-regularized convex optimization algorithms for photon-limited images. A comprehensive empirical evaluation of the proposed method helps characterize the performance of this approach relative to other state-of-the-art denoising methods. The results reveal that, despite its conceptual simplicity, Poisson PCA-based denoising appears to be highly competitive in very low light regimes.

Formation, Simulation and Restoration of Hypertelescopes Images

This article first provides a historical and detailed introduction to the image formation models for diluted pupils array and their densified versions called hypertelescopes . We propose in particular an original derivation showing that densification using a periscopic setting like in Michelson’s 20 − foot interferometer, or using inverted Galilean telescopes are fully equivalent. After a review based on previous reference studies (Tallon & Tallon-Bosc 1992; Labeyrie 1996; Aime 2008 and Aime et al. 2012), the introductory part ends with a tutorial section for simulating optical interferometric images produced by cophased arrays. We illustrate in details how the optical image formation model can be used to simulate hypertelescopes images, including sampling issues and their effects on the observed images. In a second part of the article, we address the issue of restoring hypertelescope images and present numerical illustrations obtained for classical (constrained Maximum Likelihood) methods. We also provide a detailed survey of more recent deconvolution methods based on sparse representations and of their spread in interferometric image reconstruction. The last part of the article is dedicated to two original and numerical studies. The first study shows by Monte Carlo simulations that the restoration quality achieved by constrained ML methods applied to photon limited images obtained from a diluted array on a square grid, or from a densified array (without spectral aliasing) on a grid, are essentially equivalent. The second study shows that it is possible to recover in hypertelescopes images quasi point sources that are not only far outside the clean field, but also superimposed on the replicas of other objects. This is true at least for the considered pupil array and in the limit of vanishing noise.

Distance Extraction by Means of Photon-Counting Passive Sensing Combined with Integral Imaging

Photon-counting sensing is a widely used technique for low-light-level imaging applications. This paper proposes a distance information extraction method with photon-counting passive sensing under low-lightlevel conditions. The photo-counting passive sensing combined with integral imaging generates a photon-limited elemental image array. Maximum-likelihood estimation (MLE) is used to reconstruct the photon-limited image at certain depth levels. The distance information is extracted at the depth level that minimizes the sum of the standard deviation of the corresponding photo-events in the elemental image array. Experimental and simulation results confirm that the proposed method can extract the distance information of the object under low-light-level conditions.

Bayesian Smoothing of Photon-Limited Images with Applications in Astronomy

Summary We consider a multiscale model for intensities in photon-limited images using a Bayesian framework. A typical Dirichlet prior on relative intensities is not efficient in picking up structures owing to the continuity of intensities. We propose a novel prior using the so-called ‘Chinese restaurant process’ to create structures in the form of equal intensities of some neighbouring pixels. Simulations are conducted using several photon-limited images, which are common in X-ray astronomy and other high energy photon-based images. Applications to astronomical images from the Chandra X-ray Observatory satellite are shown. The new methodology outperforms most existing methods in terms of image processing quality, speed and the ability to select smoothing parameters automatically.

Bayesian Inference on Multiscale Models for Poisson Intensity Estimation: Applications to Photon-Limited Image Denoising

We present an improved statistical model for analyzing Poisson processes, with applications to photon-limited imaging. We build on previous work, adopting a multiscale representation of the Poisson process in which the ratios of the underlying Poisson intensities (rates) in adjacent scales are modeled as mixtures of conjugate parametric distributions. Our main contributions include: 1) a rigorous and robust regularized expectation-maximization (EM) algorithm for maximum-likelihood estimation of the rate-ratio density parameters directly from the noisy observed Poisson data (counts); 2) extension of the method to work under a multiscale hidden Markov tree model (HMT) which couples the mixture label assignments in consecutive scales, thus modeling interscale coefficient dependencies in the vicinity of image edges; 3) exploration of a 2-D recursive quad-tree image representation, involving Dirichlet-mixture rate-ratio densities, instead of the conventional separable binary-tree image representation involving beta-mixture rate-ratio densities; and 4) a novel multiscale image representation, which we term Poisson-Haar decomposition, that better models the image edge structure, thus yielding improved performance. Experimental results on standard images with artificially simulated Poisson noise and on real photon-limited images demonstrate the effectiveness of the proposed techniques.

Denoising for 3-D Photon-Limited Imaging Data Using Nonseparable Filterbanks

In this paper, we present a novel frame-based denoising algorithm for photon-limited 3-D images. We first construct a new 3-D nonseparable filterbank by adding elements to an existing frame in a structurally stable way. In contrast with the traditional 3-D separable wavelet system, the new filterbank is capable of using edge information in multiple directions. We then propose a data-adaptive hysteresis thresholding algorithm based on this new 3-D nonseparable filterbank. In addition, we develop a new validation strategy for denoising of photon-limited images containing sparse structures, such as neurons (the structure of interest is less than 5% of total volume). The validation method, based on tubular neighborhoods around the structure, is used to determine the optimal threshold of the proposed denoising algorithm. We compare our method with other state-of-the-art methods and report very encouraging results on applications utilizing both synthetic and real data.

Photon Counting Linear Discriminant Analysis with Integral Imaging for Occluded Target Recognition

This paper discusses a photon-counting linear discriminant analysis (LDA) with computational integral imaging (II). The computational II method reconstructs three-dimensional (3D) objects on the reconstruction planes located at arbitrary depth-levels. A maximum likelihood estimation (MLE) can be used to estimate the Poisson parameters of photon counts in the reconstruction space. The photon-counting LDA combined with the computational II method is developed in order to classify partially occluded objects with photon-limited images. Unknown targets are classified with the estimated Poisson parameters while reconstructed irradiance images are trained. It is shown that a low number of photons are sufficient to classify occluded objects with the proposed method.

Image Reconstruction and Multidimensional Field Estimation From Randomly Scattered Sensors

Many important problems in statistical signal processing can be formulated as function estimation from randomly scattered sensors in a multidimensional space, e.g., image reconstruction from photon-limited images and field estimation from scattered sensors. We present a novel approach to the study of signal reconstruction from random samples in a multidimensional space. In particular, we study a classical iterative reconstruction method and demonstrate that it forms a sequence of unbiased estimates for band-limited signals, which converge to the true function in the mean-square sense. We subsequently rely on the iterative estimation method for multidimensional image reconstruction and field estimation from sensors scattered according to a multidimensional Poisson and uniform distribution. Computer simulation experiments are used to demonstrate the efficiency of the iterative estimation method in image reconstruction and field estimation from randomly scattered sensors.