Single-photon Avalanche Diode Camera Research Articles

A time-correlated single photon counting SPAD array camera with a bespoke data-processing algorithm for lightsheet fluorescence lifetime imaging (FLIM) and FLIM videos

A wide-field microscope with epi-fluorescence and selective plane illumination was combined with a single-photon avalanche diode (SPAD) array camera to enable live-cell fluorescence lifetime imaging (FLIM) using time-correlated single-photon counting (TCSPC). The camera sensor comprised of 192×128\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {192}\ imes \ ext {128}$$\\end{document} pixels, each integrating a single SPAD and a time-to-digital converter. Jointly, they produced a stream of single-photon images of photon arrival times with ≈38 ps\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx \ ext {38 ps}$$\\end{document} accuracy. The photon arrival times were subject to systematic delays and nonlinearities, which were corrected by a Monte-Carlo algorithm. The SPAD camera was then applied to FLIM where histogramming the resulting photon arrival times in each pixel resulted in decays compatible with common data processing pipelines for fluorescence lifetime analysis. The capabilities of the TCSPC camera-based FLIM microscope were demonstrated by imaging living unicellular photosynthetic algae and artificial lipid vesicles. Epi-fluorescence illumination enabled rapid fluorescence lifetime imaging of living cells and selective-plane illumination enabled 3-dimensional FLIM of stationary samples.

Subsurface fluorescence time-of-flight imaging using a large-format single-photon avalanche diode sensor for tumor depth assessment.

Fluorescence guidance is used clinically by surgeons to visualize anatomical and/or physiological phenomena in the surgical field that are difficult or impossible to detect by the naked eye. Such phenomena include tissue perfusion or molecular phenotypic information about the disease being resected. Conventional fluorescence-guided surgery relies on long, microsecond scale laser pulses to excite fluorescent probes. However, this technique only provides two-dimensional information; crucial depth information, such as the location of malignancy below the tissue surface, is not provided. We developed a depth sensing imaging technique using light detection and ranging (LiDAR) time-of-flight (TOF) technology to sense the depth of target tissue while overcoming the influence of tissue optical properties and fluorescent probe concentration. The technology is based on a large-format (), binary, gated, single-photon avalanche diode (SPAD) sensor with an 18ps time-gate step, synchronized with a picosecond pulsed laser. The fast response of the sensor was developed and tested for its ability to quantify fluorescent inclusions at depth and optical properties in tissue-like phantoms through analytical model fitting of the fast temporal remission data. After calibration and algorithmic extraction of the data, the SPAD LiDAR technique allowed for sub-mm resolution depth sensing of fluorescent inclusions embedded in tissue-like phantoms, up to a maximum of 5mm in depth. The approach provides robust depth sensing even in the presence of variable tissue optical properties and separates the effects of fluorescence depth from absorption and scattering variations. LiDAR TOF fluorescence imaging using an SPAD camera provides both fluorescence intensity images and the temporal profile of fluorescence, which can be used to determine the depth at which the signal is emitted over a wide field of view. The proposed tool enables fluorescence imaging at a higher depth in tissue and with higher spatial precision than standard, steady-state fluorescence imaging tools, such as intensity-based near-infrared fluorescence imaging, optical coherence tomography, Raman spectroscopy, or confocal microscopy. Integration of this technique into a standard surgical tool could enable rapid, more accurate estimation of resection boundaries, thereby improving the surgeon's efficacy and efficiency, and ultimately improving patient outcomes.

LinoSPAD2: an FPGA-based, hardware-reconfigurable 512×1 single-photon camera system.

We report on LinoSPAD2, a single-photon camera system, comprising a 512×1 single-photon avalanche diode (SPAD) front-end and one or two FPGA-based back-ends. Digital signals generated by the SPADs are processed by the FPGA in real time, whereas the FPGA offers full reconfigurability at a very high level of granularity both in time and space domains. The LinoSPAD2 camera system can process 512 SPADs simultaneously through 256 channels, duplicated on each FPGA-based back-end, with a bank of 64 time-to-digital converters (TDCs) operating at 133 MSa/s, whereas each TDC has a time resolution of 20 ps (LSB). To the best of our knowledge, LinoSPAD2 is the first fully reconfigurable SPAD camera system of large format. The SPAD front-end features a pitch of 26.2 μm, a native fill factor of 25.1%, and a microlens array achieving 2.3× concentration factor. At room temperature, the median dark count rate (DCR) is 80 cps at 7 V excess bias, the peak photon detection probability (PDP) is 53% at 520 nm wavelength, and the single-photon timing resolution (SPTR) is 50 ps FWHM. The instrument response function (IRF) is around 100 ps FWHM at system level. The LinoSPAD2 camera system is suitable for numerous applications, including LiDAR imaging, heralded spectroscopy, compressive Raman sensing, and other computational imaging techniques.

Multi-wavelength SPAD photoplethysmography for cardio-respiratory monitoring

There is a growing interest in photoplethysmography (PPG) for the continuous monitoring of cardio-respiratory signals by portable instrumentation aimed at the early diagnosis of cardiovascular diseases. In this context, it is conceivable that PPG sensors working at different wavelengths simultaneously can optimize the identification of apneas and the quantification of the associated heart-rate changes or other parameters that depend on the PPG shape (e.g., systematic vascular resistance and pressure), when evaluating the severity of breathing disorders during sleep and in general for health monitoring. Therefore, the objective of this work is to present a novel pulse oximeter that provides synchronous data logging related to three light wavelengths (green, red, and infrared) in transmission mode to optimize both heart rate measurements and a reliable and continuous assessment of oxygen saturation. The transmission mode is considered more robust over motion artifacts than reflection mode, but current pulse oximeters cannot employ green light in transmission mode due to the high absorbance of body tissues at this wavelength. For this reason, our device is based on a Single-Photon Avalanche Diode (SPAD) with very short deadtime (less than 1 ns) to have, at the same time, the single photon sensitivity and high-count rate that allows acquiring all the wavelengths of interest on the same site and in transmission mode. Previous studies have shown that SPAD cameras can be used for measuring the heart rate through remote PPG, but oxygen saturation and heart-rate measures through contact SPAD-based PPG sensors have never been addressed so far. The results of the preliminary validation on six healthy volunteers reflect the expected physiological phenomena, providing rms errors in the Inter Beat Interval estimation smaller than 70 ms (with green light) and a maximum error in the oxygen saturation smaller than 1% during the apneas. Our prototype demonstrates the reliability of SPAD-based devices for continuous long-term monitoring of cardio-respiratory variables as an alternative to photodiodes especially when minimal area and optical power are required.

Light-sheet autofluorescence lifetime imaging with a single-photon avalanche diode array.

Fluorescence lifetime imaging microscopy (FLIM) of the metabolic co-enzyme nicotinamide adenine dinucleotide (phosphate) [NAD(P)H] is a popular method to monitor single-cell metabolism within unperturbed, living 3D systems. However, FLIM of NAD(P)H has not been performed in a light-sheet geometry, which is advantageous for rapid imaging of cells within live 3D samples. We aim to design, validate, and demonstrate a proof-of-concept light-sheet system for NAD(P)H FLIM. A single-photon avalanche diode camera was integrated into a light-sheet microscope to achieve optical sectioning and limit out-of-focus contributions for NAD(P)H FLIM of single cells. An NAD(P)H light-sheet FLIM system was built and validated with fluorescence lifetime standards and with time-course imaging of metabolic perturbations in pancreas cancer cells with 10s integration times. NAD(P)H light-sheet FLIM in vivo was demonstrated with live neutrophil imaging in a larval zebrafish tail wound also with 10s integration times. Finally, the theoretical and practical imaging speeds for NAD(P)H FLIM were compared across laser scanning and light-sheet geometries, indicating a to acquisition speed advantage for the light sheet compared to the laser scanning geometry. FLIM of NAD(P)H is feasible in a light-sheet geometry and is attractive for 3D live cell imaging applications, such as monitoring immune cell metabolism and migration within an organism.

Field programmable gate array compression for large array multispeckle diffuse correlation spectroscopy.

Diffuse correlation spectroscopy (DCS) is an indispensable tool for quantifying cerebral blood flow noninvasively by measuring the autocorrelation function (ACF) of the diffused light. Recently, a multispeckle DCS approach was proposed to scale up the sensitivity with the number of independent speckle measurements, leveraging the rapid development of single-photon avalanche diode (SPAD) cameras. However, the extremely high data rate from advanced SPAD cameras is beyond the data transfer rate commonly available and requires specialized high-performance computation to calculate large number of autocorrelators (ACs) for real-time measurements. We aim to demonstrate a data compression scheme in the readout field-programmable gate array (FPGA) of a large-pixel-count SPAD camera. On-FPGA, data compression should democratize SPAD cameras and streamline system integration for multispeckle DCS. We present a SPAD array with 128 linear ACs embedded on an FPGA to calculate 12,288 ACFs in real time. We achieved a signal-to-noise ratio (SNR) gain of 110 over a single-pixel DCS system and more than threefold increase in SNR with respect to the state-of-the-art multispeckle DCS. The FPGA-embedded autocorrelation algorithm offers a scalable data compression method to large SPAD array, which can improve the sensitivity and usability of multispeckle DCS instruments.

Resolution and penetration depth of reflection-mode time-domain near infrared optical tomography using a ToF SPAD camera.

In a turbid medium such as biological tissue, near-infrared optical tomography (NIROT) can image the oxygenation, a highly relevant clinical parameter. To be an efficient diagnostic tool, NIROT has to have high spatial resolution and depth sensitivity, fast acquisition time, and be easy to use. Since many tissues cannot be penetrated by near-infrared light, such tissue needs to be measured in reflection mode, i.e., where light emission and detection components are placed on the same side. Thanks to the recent advance in single-photon avalanche diode (SPAD) array technology, we have developed a compact reflection-mode time-domain (TD) NIROT system with a large number of channels, which is expected to substantially increase the resolution and depth sensitivity of the oxygenation images. The aim was to test this experimentally for our SPAD camera-empowered TD NIROT system. Experiments with one and two inclusions, i.e., optically dense spheres of 5mm radius, immersed in turbid liquid were conducted. The inclusions were placed at depths from 10mm to 30mm and moved across the field-of-view. In the two-inclusion experiment, two identical spheres were placed at a lateral distance of 8mm. We also compared short exposure times of 1s, suitable for dynamic processes, with a long exposure of 100s. Additionally, we imaged complex geometries inside the turbid medium, which represented structural elements of a biological object. The quality of the reconstructed images was quantified by the root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), and dice similarity. The two small spheres were successfully resolved up to a depth of 30mm. We demonstrated robust image reconstruction even at 1s exposure. Furthermore, the complex geometries were also successfully reconstructed. The results demonstrated a groundbreaking level of enhanced performance of the NIROT system based on a SPAD camera.

Comparison of super-resolution and noise reduction for passive single-photon imaging

Single-photon sensitive image sensors have recently gained popularity in passive imaging applications where the goal is to capture photon flux (brightness) values of different scene points in the presence of challenging lighting conditions and scene motion. Recent work has shown that high-speed bursts of single-photon timestamp information captured using a single-photon avalanche diode camera can be used to estimate and correct for scene motion thereby improving signal-to-noise ratio and reducing motion blur artifacts. We perform a comparison of various design choices in the processing pipeline used for noise reduction, motion compensation, and upsampling of single-photon timestamp frames. We consider various pixelwise noise reduction techniques in combination with state-of-the-art deep neural network upscaling algorithms to super-resolve intensity images formed with single-photon timestamp data. We explore the trade space of motion blur and signal noise in various scenes with different motion content. Using real data captured with a hardware prototype, we achieved super-resolution reconstruction at frame rates up to 65.8 kHz (native sampling rate of the sensor) and captured videos of fast-moving objects. The best reconstruction is obtained with the motion compensation approach, which achieves a structural similarity (SSIM) of about 0.67 for fast-moving rigid objects. We are able to reconstruct subpixel resolution. These results show the relative superiority of our motion compensation compared to other approaches that do not exceed an SSIM of 0.5.

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera.

Near-infrared (NIR) fluorescence lifetime imaging (FLI) provides a unique contrast mechanism to monitor biological parameters and molecular events in vivo. Single-photon avalanche diode (SPAD) cameras have been recently demonstrated in FLI microscopy (FLIM) applications, but their suitability for in vivo macroscopic FLI (MFLI) in deep tissues remains to be demonstrated. Herein, we report in vivo NIR MFLI measurement with SwissSPAD2, a large time-gated SPAD camera. We first benchmark its performance in well-controlled in vitro experiments, ranging from monitoring environmental effects on fluorescence lifetime, to quantifying Förster resonant energy transfer (FRET) between dyes. Next, we use it for in vivo studies of target-drug engagement in live and intact tumor xenografts using FRET. Information obtained with SwissSPAD2 was successfully compared to that obtained with a gated intensified charge-coupled device (ICCD) camera, using two different approaches. Our results demonstrate that SPAD cameras offer a powerful technology for in vivo preclinical applications in the NIR window.

Quantum microscopy based on Hong–Ou–Mandel interference

Hong-Ou-Mandel (HOM) interference, the bunching of indistinguishable photons at a beam splitter, is a staple of quantum optics and lies at the heart of many quantum sensing approaches and recent optical quantum computers. Here, we report a full-field, scan-free, quantum imaging technique that exploits HOM interference to reconstruct the surface depth profile of transparent samples. We demonstrate the ability to retrieve images with micrometre-scale depth features with a photon flux as small as 7 photon pairs per frame. Using a single photon avalanche diode camera we measure both the bunched and anti-bunched photon-pair distributions at the HOM interferometer output which are combined to provide a lower-noise image of the sample. This approach demonstrates the possibility of HOM microscopy as a tool for label-free imaging of transparent samples in the very low photon regime.

Fast Skin Segmentation on Low-Resolution Grayscale Images for Remote PhotoPlethysmoGraphy

Facial skin segmentation is an important preliminary task in many applications, including remote PhotoPlethysmoGraphy (rPPG), which is the problem of estimating the heart activity of a subject just by analyzing a video of their face. By selecting all the subject’s skin surface, a more robust pulse signal could be extracted and analyzed in order to provide an accurate heart activity monitoring. Single-photon avalanche diode cameras have proven to be able to achieve better results in rPPG than traditional cameras. Although this kind of cameras produces accurate photon counts at high frame rate, they are able to capture just grayscale low resolution images. For this reason, in this work, we propose a novel skin segmentation method based on deep learning that is able to precisely localize skin pixels inside a low-resolution grayscale image. Moreover, since the proposed method makes use of depthwise separable convolutional layers, it could achieve real-time performances even when implemented on a small low-powered IoT device.

Quantized Fourier ptychography with binary images from SPAD cameras

Recently developed single-photon avalanche diode (SPAD) array cameras provide single-photon sensitivity and picosecond-scale time gating for time-of-flight measurements, with applications in LIDAR and fluorescence lifetime imaging. As compared to standard image sensors, SPAD arrays typically return binary intensity measurements with photon time-of-arrival information from fewer pixels. Here, we study the feasibility of implementing Fourier ptychography (FP), a synthetic aperture imaging technique, with SPAD array cameras to reconstruct an image with higher resolution and larger dynamic range from acquired binary intensity measurements. Toward achieving this goal, we present (1) an improved FP reconstruction algorithm that accounts for discretization and limited bit depth of the detected light intensity by image sensors, and (2) an illumination angle-dependent source brightness adaptation strategy, which is sample-specific. Together, these provide a high-quality amplitude and phase object reconstruction, not only from binary SPAD array intensity measurements, but also from alternative low-dynamic-range images, as demonstrated by our simulations and proof-of-concept experiments.

Combining time of flight and photometric stereo imaging for 3D reconstruction of discontinuous scenes.

Time of flight and photometric stereo are two three-dimensional (3D) imaging techniques with complementary properties, where the former can achieve depth accuracy in discontinuous scenes, and the latter can reconstruct surfaces of objects with fine depth details and high spatial resolution. In this work, we demonstrate the surface reconstruction of complex 3D fields with discontinuity between objects by combining the two imaging methods. Using commercial LEDs, a single-photon avalanche diode camera, and a mobile phone device, high resolution of surface reconstruction is achieved with a RMS error of 6% for an object auto-selected from a scene imaged at a distance of 50 cm.

Full-field quantum imaging with a single-photon avalanche diode camera

Single-photon-avalanche diode (SPAD) arrays are essential tools in biophotonics, optical ranging and sensing and quantum optics. However, their small number of pixels, low quantum efficiency and small fill factor have so far hindered their use for practical imaging applications. Here, we demonstrate full-field entangled photon pair correlation imaging using a 100-kpixels SPAD camera. By measuring photon coincidences between more than 500 million pairs of positions, we retrieve the full point spread function of the imaging system and subsequently high-resolution images of target objects illuminated by spatially entangled photon pairs. We show that our imaging approach is robust against stray light, enabling quantum imaging technologies to move beyond laboratory experiments towards real-world applications such as quantum LiDAR.

Towards real-time quantum imaging with single photon avalanche diode cameras

By harnessing the properties of photonic quantum states and their interaction with the environment, quantum imaging promises to go beyond the limits of classical imaging. However, the inherent weakness of detected signals and the fragility of quantum states make their properties difficult to measure in practice. In recent years, the emergence of single-photon sensitive cameras enabled the field to take a step closer to practical applications. In this respect, singlephoton avalanche diode (SPAD) cameras are one the most promising technologies as they can detect single photons across many pixels with unparalleled speed, temporal resolution, and very low noise.

Imaging and certifying high-dimensional entanglement with a single-photon avalanche diode camera

Spatial correlations between two photons are the key resource in realising many quantum imaging schemes. Measurement of the bi-photon correlation map is typically performed using single-point scanning detectors or single-photon cameras based on charged coupled device (CCD) technology. However, both approaches are limited in speed due to the slow scanning and the low frame rate of CCD-based cameras, resulting in data acquisition times on the order of many hours. Here, we employ a high frame rate, single-photon avalanche diode (SPAD) camera, to measure the spatial joint probability distribution of a bi-photon state produced by spontaneous parametric down-conversion, with statistics taken over 107 frames. Through violation of an Einstein–Podolsky–Rosen criterion by 227 sigmas, we confirm the presence of spatial entanglement between our photon pairs. Furthermore, we certify, in just 140 s, an entanglement dimensionality of 48. Our work demonstrates the potential of SPAD cameras in the rapid characterisation of photonic entanglement, leading the way towards real-time quantum imaging and quantum information processing.

Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation

Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080—human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.

High-sensitivity multispeckle diffuse correlation spectroscopy.

.Significance: Cerebral blood flow is an important biomarker of brain health and function as it regulates the delivery of oxygen and substrates to tissue and the removal of metabolic waste products. Moreover, blood flow changes in specific areas of the brain are correlated with neuronal activity in those areas. Diffuse correlation spectroscopy (DCS) is a promising noninvasive optical technique for monitoring cerebral blood flow and for measuring cortex functional activation tasks. However, the current state-of-the-art DCS adoption is hindered by a trade-off between sensitivity to the cortex and signal-to-noise ratio (SNR).Aim: We aim to develop a scalable method that increases the sensitivity of DCS instruments.Approach: We report on a multispeckle DCS (mDCS) approach that is based on a 1024-pixel single-photon avalanche diode (SPAD) camera. Our approach is scalable to independent speckle measurements since large-pixel-count SPAD cameras are becoming available, owing to the investments in LiDAR technology for automotive and augmented reality applications.Results: We demonstrated a 32-fold increase in SNR with respect to traditional single-speckle DCS.Conclusion: A mDCS system that is based on a SPAD camera serves as a scalable method toward high-sensitivity DCS measurements, thus enabling both high sensitivity to the cortex and high SNR.

Dynamic time domain near-infrared optical tomography based on a SPAD camera.

In many clinical applications it is relevant to observe dynamic changes in oxygenation. Therefore the ability of dynamic imaging with time domain (TD) near-infrared optical tomography (NIROT) will be important. But fast imaging is a challenge. The data acquisition of our handheld TD NIROT system based on single photon avalanche diode (SPAD) camera and 11 light sources was consequently accelerated. We tested the system on a diffusive medium simulating tissue with a moving object embedded. With 3D image reconstruction, the moving object was correctly located using only 0.2 s exposure time per source.

Computational 4D imaging of light-in-flight with relativistic effects

Light-in-flight imaging enables the visualization and characterization of light propagation, which provides essential information for the study of the fundamental phenomena of light. A camera images an object by sensing the light emitted or reflected from it, and interestingly, when a light pulse itself is to be imaged, the relativistic effects, caused by the fact that the distance a pulse travels between consecutive frames is of the same scale as the distance that scattered photons travel from the pulse to the camera, must be accounted for to acquire accurate space–time information of the light pulse. Here, we propose a computational light-in-flight imaging scheme that records the projection of light-in-flight on a transverse x−y plane using a single-photon avalanche diode camera, calculates z and t information of light-in-flight via an optical model, and therefore reconstructs its accurate ( x , y , z , t ) four-dimensional information. The proposed scheme compensates the temporal distortion in the recorded arrival time to retrieve the accurate time of a light pulse, with respect to its corresponding spatial location, without performing any extra measurements. Experimental light-in-flight imaging in a three-dimensional space of 375 mm × 75 mm × 50 mm is performed, showing that the position error is 1.75 mm, and the time error is 3.84 ps despite the fact that the camera time resolution is 55 ps, demonstrating the feasibility of the proposed scheme. This work provides a method to expand the recording and measuring of repeatable transient events with extremely weak scattering to four dimensions and can be applied to the observation of optical phenomena with ps temporal resolution.