Low Photon Count Research Articles

Low-Photon Counts Coherent Modulation Imaging via Generalized Alternating Projection Algorithm

Phase contrast imaging is advantageous for mitigating radiation damage to samples, such as biological specimens. For imaging at nanometer or atomic resolution, the required flux on samples increases dramatically and can easily exceed the sample damage threshold. Coherent modulation imaging (CMI) can provide quantitative absorption and phase images of samples at diffraction-limited resolution with fast convergence. When used for radiation-sensitive samples, CMI experiments need to be conducted under low illumination flux for high resolution. Here, an algorithmic framework is proposed for CMI involving generalized alternating projection and total variation constraint. A five-to-ten-fold lower photon requirement can be achieved for near-field or far-field experiment dataset. The work would make CMI more applicable to the dynamics study of radiation-sensitive samples.

Image Recovery from Synthetic Noise Artifacts in CT Scans Using Modified U-Net.

Computed Tomography (CT) is commonly used for cancer screening as it utilizes low radiation for the scan. One problem with low-dose scans is the noise artifacts associated with low photon count that can lead to a reduced success rate of cancer detection during radiologist assessment. The noise had to be removed to restore detail clarity. We propose a noise removal method using a new model Convolutional Neural Network (CNN). Even though the network training time is long, the result is better than other CNN models in quality score and visual observation. The proposed CNN model uses a stacked modified U-Net with a specific number of feature maps per layer to improve the image quality, observable on an average PSNR quality score improvement out of 174 images. The next best model has 0.54 points lower in the average score. The score difference is less than 1 point, but the image result is closer to the full-dose scan image. We used separate testing data to clarify that the model can handle different noise densities. Besides comparing the CNN configuration, we discuss the denoising quality of CNN compared to classical denoising in which the noise characteristics affect quality.

A built-in self-test module for 16-bit parallel photon counting circuit using 180 nm CMOS process

This study investigated the use of a built-in-self-test (BIST) module detecting catastrophic errors in photon-counter accumulator for liquid contamination level measurement. Efficient algorithms are exceptionally demanded for a high-count rate and low voltage system photon counting circuit on-chip. The photon counter sensors are also required high sensitivity digital counter that encodes the arrival of photon in precise timing to prevent any count erroring the absence of light. The proposed BIST is integrated on the data acquisition system, where the accumulator is located. The design circuit, functionality and topology tests of BIST and circuit under test are realized with 180 nm Silterra CMOS Process. The same Verilog codes are verified using field programmable gate array (FPGA) to predict the hardware functionality prior fabrication. The measurement was able to detect at least 90 % fault coverage within 16-bit data acquisition system at minimum operating frequency of 166.7 MHz.

LSR: Lightening super-resolution deep network for low-light image enhancement

In long-range shooting during low-light, we can only obtain a Low-Resolution (LR) image with poor visibility due to the limitation of physical devices. To get high-quality distant images, optical lenses are the optimal choice, but they are quite expensive and bulky. Low-light enhancement and Super-Resolution (SR) are the necessity of mobile phones and surveillance cameras and have wide applications in video surveillance, remote sensing, and night photography. Usually, the images that are taken from long-range in low-light suffer the loss of details not only due to low photon count but also due to low Signal-to-Noise Ratio (SNR), which makes it a highly ill-posed problem. Therefore, to resolve these two problems simultaneously, we propose a Lightening Super-Resolution (LSR) deep network. The proposed network uses the back-projection to learn the enhanced and dark features in low-resolution space iteratively and up-sample the enhanced features at the last stage of the network to get the final enhanced and high-resolution image. In particular, to train the network, the low-light images of the publicly available LOw Light (LOL) dataset are down-sampled using bicubic interpolation, and ground truth images are used as enhanced high-resolution images. For a fair comparison, a series of already available SR networks have been trained on the mentioned dataset, and their performances are compared. The promising results open up many opportunities for future work.

Material classification from sparse spectral X-ray CT using vectorial total variation based on L infinity norm

The development of energy resolving photon counting detectors (PCD) has paved the way to spectral X-ray Computed Tomography (CT), with which one can simultaneously extract the energy dependence of a material's linear attenuation coefficient (LAC). Spectral CT has proved to be an advanced technique to classify materials based on their physical properties such as electron density ( ρ e ) and effective atomic number ( Z eff ). However, the application of spectral CT may be hindered by the poor image reconstruction quality when used for material identification in security screening where rapid scanning is required. Image reconstruction from few projections or low radiation exposure time would enable rapid scanning. The reconstruction quality for each energy bin may also be degraded since the division of photon counts into multiple energy bins naturally leads to higher noise levels. In this work, we explore how to perform accurate spectral CT reconstructions from such data, and propose L ∞ norm-based vectorial total variation (L ∞ -VTV) regularization that uses correlations between multiple energy bins. Using experimental data acquired with a custom laboratory instrument for spectral CT, the L ∞ -VTV is tested on “real life” phantoms consisting of materials in the range of 6 ≤ Z eff ≤ 15. For material classification from only 7 projections, the L ∞ -VTV gives the relative deviations of 3.5% for ρ e and 2.4% for Z eff , whereas the total nuclear variation (TNV) of another state-of-the-art joint reconstruction and the total variation (TV) yield those of 3.4% and 3.1%, and 3.8% and 4.2%, respectively. The L ∞ -VTV is now ready for use in security screening. • A joint reconstruction algorithm with L infinity norm improves image reconstruction. • The algorithm is robust to few projections and/or low photon counts. • Using correlations between multiple energy bins enhances material identification. • L infinity norm in the algorithm leads to suppression of outliers in gradient magnitudes. • High noise levels in specific energy bins can be alleviated using unaffected bins.

Improving throughput and image quality of high-resolution 3D X-ray microscopes using deep learning reconstruction techniques

In high-resolution X-ray computed tomography (CT), also known as 3D X-ray microscopy (XRM), low photon counts can lead to extremely long data acquisition times (in the order of hours). Reducing the number of radiographic projections (Np) acquired for CT reconstruction can be a cost-efficient solution in some cases. But the risk associated with reducing Np, if analytical filteredbackprojection algorithms are used for CT reconstruction, e.g., Feldkamp-Davis-Kress (FDK), is that it may produce a significant loss of image quality. Typical Np thresholds for a faithful 3D image reconstruction, required by the Nyquist-Shannon sampling theorem, are in the order of thousand projection views with modern XRM instruments. It is now well known, however, that deep learning (DL) based algorithms for CT reconstruction can improve the scan time (throughput) and image quality capabilities of XRM. This paper proposes the use of DL-based algorithms as an option for reducing Np, even down to a few hundred projections, without a significant loss of image quality. The integration of DL-based reconstruction techniques into 3D XRM workflows is presented throughout this article. It is shown that 3D XRM data reconstructions produced by DL-based workflows can provide up to 8X and 10X throughput improvement at similar or better image quality compared to standard FDK reconstruction.

Real-time magnetometry with coherent population trapping in a nitrogen-vacancy center

We demonstrate real-time magnetometry by detecting fluorescence from a nitrogen-vacancy center in the setting of coherent population trapping and by estimating the magnetic field from the time series of the observed photon counts, which are correlated with the underlying field variation. The proof-of-principle experiment uses an external time-varying magnetic field that follows an Ornstein-Uhlenbeck (OU) process. A conventional estimator based on the use of average photon counts per relevant time interval is susceptible to photon number shot noise and performs poorly at relatively low average photon counts. In comparison, by taking into consideration the statistical properties of the OU process, a Bayesian inference-based estimator can effectively update dynamical information of the field in real time with the detection of just a single photon.

Boosting Photon-Efficient Image Reconstruction With A Unified Deep Neural Network.

Photon-efficient imaging, which captures 3D images with single-photon sensors, has enabled a wide range of applications. However, two major challenges limit the reconstruction performance, i.e., the low photon counts accompanied by low signal-to-background ratio (SBR) and the multiple returns. In this paper, we propose a unified deep neural network that, for the first time, explicitly addresses these two challenges, and simultaneously recovers depth maps and intensity images from photon-efficient measurements. Starting from a general image formation model, our network is constituted of one encoder, where a non-local block is utilized to exploit the long-range correlations in both spatial and temporal dimensions of the raw measurement, and two decoders, which are designed to recover depth and intensity, respectively. Meanwhile, we investigate the statistics of the background noise photons and propose a noise prior block to further improve the reconstruction performance. The proposed network achieves decent reconstruction fidelity even under extremely low photon counts / SBR and heavy blur caused by the multiple-return effect, which significantly surpasses the existing methods. Moreover, our network trained on simulated data generalizes well to real-world imaging systems, which greatly extends the application scope of photon-efficient imaging in challenging scenarios with a strict limit on optical flux. Code is available at https://github.com/JiayongO-O/PENonLocal.

ColorPolarNet: Residual Dense Network-Based Chromatic Intensity-Polarization Imaging in Low-Light Environment

Polarization imaging provides more dimensional information than traditional intensity imaging and has been widely used in both military and civil areas, such as battlefield reconnaissance, environmental monitoring, autonomous driving and so on. With the rapid development of color polarization imaging sensors, polarization detection that allows simultaneous acquisition of multiple images (intensity maps modulated by different polarizers) and real-time extraction of polarization information is becoming increasingly mature. However, the imaging quality of color polarization camera is always unsatisfactory in low light environment with low-photon count. In this paper, we present a residual dense blocks-based multi-task convolutional network, called ColorPolarNet, to extract and enhance the intensity and polarization information from four RGB polarized light intensity images captured in low light environment with different polarization orientations (0°,45°,90° and 135°). This proposed network consists of two parts: (1) an intensity network for initial denoising and color deviation correction of the four polarized light intensity maps; (2) a polarization difference network used to enhance the details of the total intensity ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> ) map, the degree of linear polarization (DoLP) map and the angle of polarization (AoP) map. Also, to train the model, we collect a low light chromatic polarization (LLCP) dataset that contains 300 paired datasets of low-normal light images captured indoors and outdoors. Both qualitative and quantitative experimental results show that the proposed ColorPolarNet outperforms other conventional low light image enhancement methods in terms of signal fidelity, contrast enhancement and color reproduction. Our codes and the LLCP dataset are available at https://github.com/MinjieWan/ColorPolarNet.

Lifetime-based analysis of binary fluorophores mixtures in the low photon count limit

SummarySingle biomolecule sensing often requires the quantification of multiple fluorescent species. Here, we theoretically and experimentally use time-resolved fluorescence via Time Correlated Single Photon Counting (TCSPC) to accurately quantify fluorescent species with similar chromatic signatures. A modified maximum likelihood estimator is introduced to include two fluorophore species, with compensation of the instrument response function. We apply this algorithm to simulated data of a simplified two-fluorescent species model, as well as to experimental data of fluorophores' mixtures and to a model protein, doubly labeled with different fluorophores' ratio. We show that 100 to 200 photons per fluorophore, in a 10-ms timescale, are sufficient to provide an accurate estimation of the dyes' ratio on the model protein. Our results provide estimation for the desired photon integration time toward implementation of TCSPC in systems with fast occurring events, such as translocation of biomolecules through nanopores or single-molecule burst analyses experiments.

Image Denoising Using Nonlocal Means with Shape-Adaptive Patches and New Weights

Digital images captured from CMOS/CCD image sensors are prone to noise due to inherent electronic fluctuations and low photon count. To efficiently reduce the noise in the image, a novel image denoising strategy is proposed, which exploits both nonlocal self-similarity and local shape adaptation. With wavelet thresholding, the residual image in method noise, derived from the initial estimate using nonlocal means (NLM), is exploited further. By incorporating the role of both the initial estimate and the residual image, spatially adaptive patch shapes are defined, and new weights are calculated, which thus results in better denoising performance for NLM. Experimental results demonstrate that our proposed method significantly outperforms original NLM and achieves competitive denoising performance compared with state-of-the-art denoising methods.

Reduction of streak artifacts caused by low photon counts utilizing an image-based forward projection in computed tomography

BackgroundThe streak artifacts in computed tomography (CT) images caused by low photon counts are known to be effectively suppressed by raw-data-based techniques. This study aims to propose a technique to reduce the streak artifact without accessing the raw data. MethodsThe proposed streak artifact reduction (SAR) technique consists of three steps: numerical forward projection to a CT image, adaptive filtering of the generated sinogram, and image reconstruction from the processed sinogram. The authors have expanded the two-dimensional method (2D-SAR) to three dimensions (3D-SAR) by using consecutive CT images. The modulation transfer function (MTF), the image noise (standard deviation), and the visibility of comb-shaped objects were evaluated at a low dose of 5 mGy. Using anthropomorphic abdominal and chest phantoms, CT images and the artifact index (AI) were compared between 3D-SAR and two types of iterative reconstruction (IR). ResultsSufficient artifact reductions associated with 54% and 61% reduction of noise for 2D- and 3D-SAR, respectively, were obtained in the phantom images, although the 50%MTF decreased by 28%. The visibility of the combs was improved with both the 2D- and 3D-SAR methods. The AI results of 3D-SAR were better than one type of IR and almost equal to the other type of IR, which was consistent with observed artifacts. ConclusionBoth 2D-SAR and 3D-SAR have turned out to be effective in reducing streak artifacts. The proposed technique will be an effective tool since it needs no raw data, and thus can be applied to any CT images produced by a wide variety of CT systems.

General coupling efficiency for fiber-fed astronomical instruments

Photonic devices (e.g., optical fibers, 2D waveguides) are commonly used on telescopes worldwide and have been hugely successful in terms of their scientific impact. Coupling light efficiently into photonic devices is critical because most astronomical sources generate low photon count rates at the detector, even when coupled to large telescope apertures. Recent developments in adaptive optics have helped to improve the coupling to optical fibers, although efficient coupling to single-mode fibers remains a challenge. We appraise the theoretical limits to this coupling efficiency in both diffraction limited and seeing limited conditions, and we review methods to improve coupling efficiency with a view to recent advances in astrophotonics.

Learning digital camera pipeline for extreme low-light imaging

In low-light conditions, a conventional camera imaging pipeline produces sub-optimal images that are usually dark and noisy due to a low photon count and low signal-to-noise ratio (SNR). We present a data-driven approach that learns the desired properties of well-exposed images and reflects them in images that are captured in extremely low ambient light environments, thereby significantly improving the visual quality of these low-light images. The recent works on this problem only consider a pixel-level loss metric that ignores perceptual quality and thus generate outputs susceptible to visual artifacts. To address this problem, we propose a new loss function that exploits the characteristics of both pixel-wise and perceptual metrics, enabling our deep neural network to learn the camera processing pipeline to transform the short-exposure, low-light RAW sensor data to well-exposed sRGB images. The results show that our method outperforms the state-of-the-art according to psychophysical tests as well as pixel-wise standard metrics and recent learning-based perceptual image quality measures. In essence, the proposed model can potentially replace the conventional digital camera pipeline for the specific case of extreme low-light imaging.

LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction

Deep learning approaches for tomographic image reconstruction have become very effective and have been demonstrated to be competitive in the field. Comparing these approaches is a challenging task as they rely to a great extent on the data and setup used for training. With the Low-Dose Parallel Beam (LoDoPaB)-CT dataset, we provide a comprehensive, open-access database of computed tomography images and simulated low photon count measurements. It is suitable for training and comparing deep learning methods as well as classical reconstruction approaches. The dataset contains over 40000 scan slices from around 800 patients selected from the LIDC/IDRI database. The data selection and simulation setup are described in detail, and the generating script is publicly accessible. In addition, we provide a Python library for simplified access to the dataset and an online reconstruction challenge. Furthermore, the dataset can also be used for transfer learning as well as sparse and limited-angle reconstruction scenarios.

A comparison of 8 and 16 frames gated SPECT imaging for determination of left ventricular volumes and ejection fraction: effects of gender and myocardial counts.

In myocardial gated SPECT imaging each cardiac cycle is divided into 8 or 16 temporal frames and the cause of the difference between 8 and 16 frames is not specified exactly. The aim of this study was to investigate the effect of myocardial detector counts and gender on the difference between 8 and 16 frames and also to compare the LVEF obtained by 8 and 16 frames with echocardiography. The study population included 84 patients who underwent gated SPECT imaging. Left ventricular parameters were assessed on 8 and 16 frames gated SPECT. LVEF was also measured with two-dimensional echocardiography within 5-10days after gated SPECT imaging. There was agood correlation between 8 and 16 frames for calculation of LVEF (p = 0.00, r = 0.860), EDV (p = 0.00, r = 0.965) and ESV (p = 0.00, r = 0.956) in all patients. But the difference between 8 and 16 frames for calculation of LVEF (p = 0.00), EDV (p = 0.014) and ESV (p = 0.00) was statistically significant. This difference was assessed separately in females, males, patients with high photon counts and patients with low photon counts and in all subgroups was statistically significant difference in the estimation of LVEF and ESV (p < 0.05) but no significant difference in the estimation of EDV (p > 0.05). Echocardiography resulted in smaller LVEF as compared to 8 and 16 frames gated SPECT studies and there was asignificant difference between thetwo methods (p = 0.00). The myocardial detector counts and gender have no effect on the difference between 8 and 16 frames methods and the LVEF on echocardiography is smaller than the gated SPECT, but the 8-frame is closer to echocardiography.

Automatic Image Segmentation Based on Phasor Space Clustering

We present our latest developments in automatic image segmentation for measurements on spectral and lifetime fluorescence imaging. The algorithm operates on the basis of the phasor approach and makes use of a well-established machine learning method for clustering N-dimensional data points, in our case the pixel data points on the phasor space. The technique is based on Gaussian mixture models solved by the use of the expectation-maximization algorithm. We show the method is the best candidate out of the many machine learning clustering techniques available due to the nature of the phasor data and also due to the spreading of points in the phasor space for the cases of low pixel photon counts. In order to quantify the power of the method we performed simulations with synthetic phasor data and we tested the method by measuring its performance against a series of established clustering techniques. We also show results on lifetime and spectral data obtained from samples of live cells. The samples were prepared by staining with a series of commercially-available organelle dyes in order to target distinct structures in the cells. We show how by taking a single spectral and lifetime acquisition we resolve up to six populations in the phasor space corresponding to the six cellular staining probes. Furthermore, we use this method in combination with our other recent work, where we make use of higher harmonics of the phasor transform in order to solve for the fractions of known components and also for lifetimes of unknown components. This multi-harmonic multi-component method is highly sensitive to noise and heterogeneous distributions and here is where our clustering method simplifies the problem by preprocessing the data and fragmenting it into pixels containing distinct lifetime/spectral characteristics. (Supported by NIH P41-GM103540).

Quantitative phase retrieval with low photon counts using an energy resolving quantum detector.

X-ray phase contrast imaging (PCI) combined with phase retrieval has the potential to improve soft-material visibility and discrimination. This work examined the accuracy, image quality gains, and robustness of a spectral phase retrieval method proposed by our group. Spectroscopic PCI measurements of a physical phantom were obtained using state-of-the-art photon-counting detectors in combination with a polychromatic x-ray source. The phantom consisted of four poorly attenuating materials. Excellent accuracy was demonstrated in simultaneously retrieving the complete refractive properties (photoelectric absorption, attenuation, and phase) of these materials. Approximately 10 times higher SNR was achieved in retrieved images compared to the original PCI intensity image. These gains are also shown to be robust against increasing quantum noise, even for acquisition times as low as 1 s with a low-flux microfocus x-ray tube (average counts of 250 photons/pixels). We expect that this spectral phase retrieval method, adaptable to several PCI geometries, will allow significant dose reduction and improved material discrimination in clinical and industrial x-ray imaging applications.

Effect of Pixelation on the Parameter Estimation of Single Molecule Trajectories.

The advent of single molecule microscopy has revolutionized biological investigations by providing a powerful tool for the study of intercellular and intracellular trafficking processes of protein molecules which was not available before through conventional microscopy. In practice, pixelated detectors are used to acquire the images of fluorescently labeled objects moving in cellular environments. Then, the acquired fluorescence microscopy images contain the numbers of the photons detected in each pixel, during an exposure time interval. Moreover, instead of having the exact locations of detection of the photons, we only know the pixel areas in which the photons impact the detector. These challenges make the analysis of single molecule trajectories, from pixelated images, a complex problem. Here, we investigate the effect of pixelation on the parameter estimation of single molecule trajectories. In particular, we develop a stochastic framework to calculate the maximum likelihood estimates of the parameters of a stochastic differential equation that describes the motion of the molecule in living cells. We also calculate the Fisher information matrix for this parameter estimation problem. The analytical results are complicated through the fact that the observation process in a microscope prohibits the use of standard Kalman filter type approaches. The analytical framework presented here is illustrated with examples of low photon count scenarios for which we rely on Monte Carlo methods to compute the associated probability distributions.

Interferometric diffuse correlation spectroscopy improves measurements at long source-detector separation and low photon count rate.

.Significance: The use of diffuse correlation spectroscopy (DCS) has shown efficacy in research studies as a technique capable of noninvasively monitoring blood flow in tissue with applications in neuromonitoring, exercise science, and breast cancer management. The ability of DCS to resolve blood flow in these tissues is related to the optical sensitivity and signal-to-noise ratio (SNR) of the measurements, which in some cases, particularly adult cerebral blood flow measurements, is inadequate in a significant portion of the population. Improvements to DCS sensitivity and SNR could allow for greater clinical translation of this technique.Aim: Interferometric diffuse correlation spectroscopy (iDCS) was characterized and compared to traditional homodyne DCS to determine possible benefits of utilizing heterodyne detection.Approach: An iDCS system was constructed by modifying a homodyne DCS system with fused fiber couplers to create a Mach–Zehnder interferometer. Comparisons between homodyne and heterodyne detection were performed using an intralipid phantom characterized at two extended source–detector separations (2.4, 3.6 cm), different photon count rates, and a range of reference arm power levels. Characterization of the iDCS signal mixing was compared to theory. Precision of the estimation of the diffusion coefficient and SNR of the autocorrelation curve were compared between different measurement conditions that mimicked what would be seen in vivo.Results: The mixture of signals present in the heterodyne autocorrelation function was found to agree with the derived theory and resulted in accurate measurement of the diffusion coefficient of the phantom. Improvement of the SNR of the autocorrelation curve up to and up to 80% reduction in the variability of the diffusion coefficient fit were observed for all measurement cases as a function of increased reference arm power.Conclusions: iDCS has the potential to improve characterization of blood flow in tissue at extended source–detector separations, enhancing depth sensitivity and SNR.