Single-photon Avalanche Diode Array Research Articles

Super-resolution time-resolved imaging using computational sensor fusion

Imaging across both the full transverse spatial and temporal dimensions of a scene with high precision in all three coordinates is key to applications ranging from LIDAR to fluorescence lifetime imaging. However, compromises that sacrifice, for example, spatial resolution at the expense of temporal resolution are often required, in particular when the full 3-dimensional data cube is required in short acquisition times. We introduce a sensor fusion approach that combines data having low-spatial resolution but high temporal precision gathered with a single-photon-avalanche-diode (SPAD) array with data that has high spatial but no temporal resolution, such as that acquired with a standard CMOS camera. Our method, based on blurring the image on the SPAD array and computational sensor fusion, reconstructs time-resolved images at significantly higher spatial resolution than the SPAD input, upsampling numerical data by a factor 12 times 12, and demonstrating up to 4 times 4 upsampling of experimental data. We demonstrate the technique for both LIDAR applications and FLIM of fluorescent cancer cells. This technique paves the way to high spatial resolution SPAD imaging or, equivalently, FLIM imaging with conventional microscopes at frame rates accelerated by more than an order of magnitude.

3D Photon-to-Digital Converter for Radiation Instrumentation: Motivation and Future Works.

Analog and digital SiPMs have revolutionized the field of radiation instrumentation by replacing both avalanche photodiodes and photomultiplier tubes in many applications. However, multiple applications require greater performance than the current SiPMs are capable of, for example timing resolution for time-of-flight positron emission tomography and time-of-flight computed tomography, and mitigation of the large output capacitance of SiPM array for large-scale time projection chambers for liquid argon and liquid xenon experiments. In this contribution, the case will be made that 3D photon-to-digital converters, also known as 3D digital SiPMs, have a potentially superior performance over analog and 2D digital SiPMs. A review of 3D photon-to-digital converters is presented along with various applications where they can make a difference, such as time-of-flight medical imaging systems and low-background experiments in noble liquids. Finally, a review of the key design choices that must be made to obtain an optimized 3D photon-to-digital converter for radiation instrumentation, more specifically the single-photon avalanche diode array, the CMOS technology, the quenching circuit, the time-to-digital converter, the digital signal processing and the system level integration, are discussed in detail.

Direct TOF Scanning LiDAR Sensor With Two-Step Multievent Histogramming TDC and Embedded Interference Filter

This article presents a 36-channel scanning light detection and ranging (LiDAR) sensor with an on-chip single-photon avalanche diode array. The sensor has an area-efficient 11-bit in situ histogramming time-to-digital converter with a 3000 ×78 μm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> per channel area based on a mixed-signal accumulator, though it is incorporated with histogramming and filtering capabilities. Furthermore, owing to its embedded interference (IF) filter, the sensor can perform reliable direct time-of-flight measurements even with IF from 32 different LiDAR sensors. The LiDAR system also has a beam scanner that comprises dual laser diodes for IF elimination and a hybrid mirror such that high-resolution images with a resolution of 2200 ×36 can be acquired with a wide field-of-view of 120 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">°</sup> ×8 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">°</sup> .

Toward high-performance SPAD arrays for space-based atmosphere and ocean profiling LiDARs

Space-based light detection and ranging (LiDAR) sensors have provided valuable insight into the global, vertical distribution of aerosol and cloud layers in Earth’s atmosphere, and, more recently, of the distribution of phytoplankton in the ocean. However, the photodetectors in these sensors lack the performance necessary to capture the vertical structure of cloud tops and ocean phytoplankton to a fidelity sufficient for advancing our understanding of the global water cycle and ocean carbon cycle, respectively. Recent advancements in high-performance single photon avalanche diode (SPAD) arrays promise to enable these measurements, while also offering a sensitivity that will allow significant reductions in laser power and telescope size, with associated sensor-level size, weight, and power (SWaP) savings. To harness the unique benefits of SPADs for these measurements, we propose to develop a large-format array of photon counting SPADs with <10 ns dead time, along with readout integrated circuitry that sums and bins (histograms) photon counts in real time to the desired temporal resolution for the target application. The feasibility of this approach has been investigated with a small-scale 8 × 8 SPAD array proof of concept hardware demonstration developed at Politecnico di Milano, with promising initial results. Progress is reported on designs that will allow scaling the array and readout integrated circuit electronics to the requisite of 128 × 128 size in a chip-scale, low power, photodetector ideal for LiDAR remote sensing of the atmosphere and ocean from SWaP-constrained platforms.

SPAD-Array Contention Signal and Noise Model Suitable for Multilevel Modulation Schemes With Signal Processing

The first signal model for a single photon avalanche diode (SPAD)-array communication receiver for multilevel modulation schemes is reported. This paper proposes a novel, generalised SPAD array signal and noise model for both digital and analogue, synchronous and asynchronous SPAD readout arrays, which includes the competition between the input photons, dark counts and after-pulsing counts. With this contention signal and noise model, multilevel signals including the signal variation after distortion or equalisation can be evaluated. Also, we report the first numerical investigation for SPAD-based, high data rate, free space, visible light communication using higher order pulse amplitude modulation (PAM) with matched filter, linear and non-linear Volterra post-equalization. Simulations have been carried out to analyze and compare the bit error rate (BER) performances under a variety of conditions. The model is verified by comparison with published experimental results.

Design of a Real-Time Breakdown Voltage and On-Chip Temperature Monitoring System for Single Photon Avalanche Diodes

The design and implementation of a real-time breakdown voltage and on-chip temperature monitoring system for single photon avalanche diodes (SPADs) is described in this work. In the system, an on-chip shaded (active area of the detector covered by a metal layer) SPAD is used to provide a dark count rate for the breakdown voltage and temperature calculation. A bias circuit was designed to provide a bias voltage scanning for the shaded SPAD. A microcontroller records the pulses from the anode of the shaded SPAD and calculates its real-time dark count rate. An algorithm was developed for the microcontroller to calculate the SPAD’s breakdown voltage and the on-chip temperature in real time. Experimental results show that the system is capable of measuring the SPAD’s breakdown voltage with a mismatch of less than 1.2%. Results also show that the system can provide real-time on-chip temperature monitoring for the range of −10 to 50 °C with errors of less than 1.7 °C. The system proposed can be used for the real-time SPAD’s breakdown voltage and temperature estimation for dual-SPADs or SPAD arrays chip where identical detectors are fabricated on the same chip and one or more dummy SPADs are shaded. With the breakdown voltage and the on-chip temperature monitoring, intelligent control logic can be developed to optimize the performance of the SPAD-based photon counting system by adjusting the parameters such as excess bias voltage and dead-time. This is particularly useful for SPAD photon counting systems used in complex working environments such as the applications in 3D LIDAR imaging for geodesy, geology, geomorphology, forestry, atmospheric physics and autonomous vehicles.

Time-Delay-Integration Imaging Implemented With Single-Photon-Avalanche-Diode Linear Array

In this paper, we propose a novel kind of time delay integration (TDI) image sensor based on single photon avalanche diode (SPAD), i.e. TDI-SPAD. Compared with existing TDI imaging devices, the proposed TDI-SPAD sensor is characterized by low noise, high sensitivity, high modulation transfer function (MTF) and transfer efficiency. The proposed architecture design of multi-stages TDI-SPAD sensor and the digital transfer circuits are discussed in detail, and the full function of TDI-imaging is demonstrated with experiments based upon two SPAD sensors. First, the imaging function of TDI-SPAD is verified through imaging of scanning scene with a $128\times128$ SPAD array. Then, we fabricate a $256\times2$ TDI-SPAD sensor, build a TDI-imaging camera system, and demonstrate the feasibility and SNR improvement of the proposed TDI-imaging sensor.

Dithered depth imaging

Single-photon lidar (SPL) is a promising technology for depth measurement at long range or from weak reflectors because of the sensitivity to extremely low light levels. However, constraints on the timing resolution of existing arrays of single-photon avalanche diode (SPAD) detectors limit the precision of resulting depth estimates. In this work, we describe an implementation of subtractively-dithered SPL that can recover high-resolution depth estimates despite the coarse resolution of the detector. Subtractively-dithered measurement is achieved by adding programmable delays into the photon timing circuitry that introduce relative time shifts between the illumination and detection that are shorter than the time bin duration. Careful modeling of the temporal instrument response function leads to an estimator that outperforms the sample mean and results in depth estimates with up to 13 times lower root mean-squared error than if dither were not used. The simple implementation and estimation suggest that globally dithered SPAD arrays could be used for high spatial- and temporal-resolution depth sensing.

Characterization of space-momentum entangled photons with a time resolving CMOS SPAD array.

Single-photon avalanche diode arrays can provide both the spatial and temporal information of each detected photon. We present here the characterization of spatially entangled photons with a 32 × 32 pixel sensor, specifically designed for quantum imaging applications. The sensor is time-tagging each detection event at pixel level with sub-nanosecond accuracy within frames of 50 ns. The spatial correlations between any number of detections in a defined temporal window can thus be directly extracted from the data.The space-momentum entanglement of photon pairs is demonstrated by violating an EPR-type inequality directly from the measured near-field correlations and far-field anti-correlations.

Characterization of single channel liquid light guide coupling and SPAD array imaging for tumour margin estimation using fluorescence lifetime

Surgery remains one of the key treatment options for tumour removal, and surgeons primarily rely on eye and touch to assess the boundary between healthy and cancerous tissue with no cellular information as guidance. There is therefore a need for a device or instrument that can be used by the surgeon in real-time during the surgical procedure to ensure as many of the cancerous cells and as few of the healthy cells have been removed as possible. Fluorescence approaches have previously demonstrated significant promise in this application, but clinical take-up has been limited and much more characterization of critical parameters needed before robotic surgery can be contemplated. Here we investigate two time-correlated single-photon counting (TCSPC) fluorescence lifetime systems for the detection of phantom tumour margins derived from silica sol-gels. A simple and low-cost liquid light guide system (LLG) incorporating a single photomultiplier detection channel and translational stage was developed. This provided a useful reference for a compact single-photon avalanche diode (SPAD) array camera system for fluorescence lifetime imaging microscopy (FLIM) which permits up to ∼25 000 in-pixel timing measurements at video rates in ambient light using only low energy (∼30 pJ) diode laser pulses to minimize cell and dye degradation. Measurements of phantom margins with sol-gel doped Rhodamine 6G (R6G) of fluorescence lifetime ∼4 ns using the LLG system demonstrates that for 7 mm excitation diameter and over 5–15 mm sol-gel LLG separation the sol-gel only region could be clearly identified 1 mm after the margin position, a widely accepted minimum surgical resolution. A comparison between measurements with the LLG and SPAD FLIM system using the sub-ns fluorescence lifetime of the FDA-approved dye indocyanine green (ICG) demonstrates that the minimum workable spatial resolution and sufficient speed are only achievable with such faster lifetimes using the SPAD FLIM system.

High-speed 3D sensing via hybrid-mode imaging and guided upsampling

Imaging systems with temporal resolution play a vital role in a diverse range of scientific, industrial, and consumer applications, e.g., fluorescent lifetime imaging in microscopy and time-of-flight (ToF) depth sensing in autonomous vehicles. In recent years, single-photon avalanche diode (SPAD) arrays with picosecond timing capabilities have emerged as a key technology driving these systems forward. Here we report a high-speed 3D imaging system enabled by a state-of-the-art SPAD sensor used in a hybrid imaging mode that can perform multi-event histogramming. The hybrid imaging modality alternates between photon counting and timing frames at rates exceeding 1000 frames per second, enabling guided upscaling of depth data from a native resolution of 64 × 32 to 256 × 128 . The combination of hardware and processing allows us to demonstrate high-speed ToF 3D imaging in outdoor conditions and with low latency. The results indicate potential in a range of applications where real-time, high throughput data are necessary. One such example is improving the accuracy and speed of situational awareness in autonomous systems and robotics.

Quanta burst photography

Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting individual incident photons, and capturing their time-of-arrival with high timing precision. While these sensors were limited to singlepixel or low-resolution devices in the past, recently, large (up to 1 MPixel) SPAD arrays have been developed. These single-photon cameras (SPCs) are capable of capturing high-speed sequences of binary single-photon images with no read noise. We present quanta burst photography, a computational photography technique that leverages SPCs as passive imaging devices for photography in challenging conditions, including ultra low-light and fast motion. Inspired by recent success of conventional burst photography, we design algorithms that align and merge binary sequences captured by SPCs into intensity images with minimal motion blur and artifacts, high signal-to-noise ratio (SNR), and high dynamic range. We theoretically analyze the SNR and dynamic range of quanta burst photography, and identify the imaging regimes where it provides significant benefits. We demonstrate, via a recently developed SPAD array, that the proposed method is able to generate high-quality images for scenes with challenging lighting, complex geometries, high dynamic range and moving objects. With the ongoing development of SPAD arrays, we envision quanta burst photography finding applications in both consumer and scientific photography.

Radiation induced degradation in a 150-nm CMOS SPADs device

The effects induced by radiation on an innovative photo-sensor consisting of Single-Photon Avalanche Diodes arrays, implemented in a 150-nm CMOS technology, are presented. Samples were irradiated with a proton beam at different doses, and the impact of radiation-induced damage on the performance of the devices in terms of Dark Count Rate has been estimated. Furthermore, possible methods to reduce the impact of radiation-induced damage are discussed.

Image reconstruction for novel time domain near infrared optical tomography: towards clinical applications.

Near infrared optical tomography (NIROT) is an emerging modality that enables imaging the oxygenation of tissue, which is a biomarker of tremendous clinical relevance. Measuring in reflectance is usually required when NIROT is applied in clinical scenarios. Single photon avalanche diode (SPAD) array technology provides a compact solution for time domain (TD) NIROT to gain huge temporal and spatial information. This makes it possible to image complex structures in tissue. The main aim of this paper is to validate the wavelength normalization method for our new TD NIROT experimentally by exposing it to a particularly difficult challenge: the recovery of two inclusions at different depths. The proposed reconstruction algorithm aims to tackle systematic errors and other artifacts with known wavelength-dependent relation. We validated the device and reconstruction method experimentally on a silicone phantom with two inclusions: one at depth of 10 mm and the other at 15 mm. Despite this tough challenge for reflectance NIROT, the system was able to localize both inclusions accurately.

Fully Integrated Time-Gated 3D Fluorescence Imager for Deep Neural Imaging.

This paper presents a device for time-gated fluorescence imaging in the deep brain, consisting of two on-chip laser diodes and 512 single-photon avalanche diodes (SPADs). The edge-emitting laser diodes deliver fluorescence excitation above the SPAD array, parallel to the imager. In the time domain, laser diode illumination is pulsed and the SPAD is time-gated, allowing a fluorescence excitation rejection up to O.D. 3 at 1ns of time-gate delay. Each SPAD pixel is masked with Talbot gratings to enable the mapping of 2D array photon counts into a 3D image. The 3D image achieves a resolution of 40, 35, and 73μm in the x, y, and z directions, respectively, in a noiseless environment, with a maximum frame rate of 50 kilo-frames-per-second. We present measurement results of the spatial and temporal profiles of the dual-pulsed laser diode illumination and of the photon detection characteristics of the SPAD array. Finally, we show the imager's ability to resolve a glass micropipette filled with red fluorescent microspheres. The system's 420μm-wide cross section allows it to be inserted at arbitrary depths of the brain while achieving a field of view four times larger than fiber endoscopes of equal diameter.

A 32 × 128 SPAD-257 TDC Receiver IC for Pulsed TOF Solid-State 3-D Imaging

A single-chip receiver for pulsed laser direct time-of-flight 3-D imaging applications has been realized in a 0.35- $\mu \text{m}$ HV CMOS technology. The chip includes a $32 \times 128$ single-photon avalanche diode (SPAD) array [35% fill factor (FF)] and 257 time-to-digital converters (TDCs) with a ~78-ps resolution. Two adjacent rows ( $2 \times 128$ SPADs) at a time can be selected for simultaneous measurement, i.e., 16 measurement cycles are needed to cover the whole array. SPADs are capable of operating in a gated mode in order to suppress dark and background light-induced detections. The IC was designed to be used in a solid-state 3-D imaging system with laser illumination concentrated in both time (short sub-ns pulses) and space (targeting only the active rows of the SPAD array). The performance of the receiver IC was characterized in a solid-state 3-D range imager with flood-pulsed illumination from a laser diode (LD)-based transmitter, which produced short [~150-ps full-width at half-maximum (FWHM)] high-energy (~3.8-nJ pulse/~14-W peak power) pulses at a pulsing rate of 250 kHz when operating at a wavelength of 810 nm. Two detector/TDC ICs formed an 8k pixel receiver, targeting a field-of-view of $\sim 42^{\circ } \times 21^{\circ }$ by means of simple optics. Frame rates of up to 20 fps were demonstrated with a centimeter-level precision in the case of Lambertian targets within a range of 3.5 m.

SPAD-based asynchronous-readout array detectors for image-scanning microscopy

Fluorescence microscopy and derived techniques are continuously looking for photodetectors able to guarantee increased sensitivity, high spatial and temporal resolution, and ease of integration into modern microscopy architectures. Recent advances in single-photon avalanche diodes (SPADs) fabricated with industry-standard microelectronic processes allow the development of new detection systems tailored to address the requirements of advanced imaging techniques (such as image-scanning microscopy). To this aim, we present the complete design and characterization of two bidimensional SPAD arrays composed of 25 fully independent and asynchronously operated pixels, both having fill factor of about 50% and specifically designed for being integrated into existing laser scanning microscopes. We used two different microelectronics technologies to fabricate our detectors: the first technology exhibiting very low noise (roughly 200 dark counts per second at room temperature) and the second one showing enhanced detection efficiency (more than 60% at a wavelength of 500 nm). Starting from the silicon-level device structures and moving towards the in-pixel and readout electronics description, we present performance assessments and comparisons between the two detectors. Images of a biological sample acquired after their integration into our custom image-scanning microscope finally demonstrate their exquisite on-field performance in terms of spatial resolution and contrast enhancement. We envisage that this work can trigger the development of a new class of SPAD-based detector arrays able to substitute the typical single-element sensor used in fluorescence laser scanning microscopy.

Observing mode-dependent wavelength-to-time mapping in few-mode fibers using a single-photon detector array

Wavelength-to-time mapping (WTM)—stretching ultrashort optical pulses in a dispersive medium such that the instantaneous frequency becomes time-dependent—is usually performed using a single-mode fiber. In a number of applications, such as time-stretch imaging (TSI), the use of this single-mode fiber during WTM limits the achievable sampling rate and the imaging quality. Multimode fiber based WTM is a potential route to overcome this challenge and project a more diverse range of light patterns. Here, we demonstrate the use of a two-dimensional single-photon avalanche diode (SPAD) array to image, in a time-correlated single-photon counting (TCSPC) manner, the time- and wavelength-dependent arrival of different spatial modes in a few-mode fiber. We then use a TCSPC spectrometer with a one-dimensional SPAD array to record and calibrate the wavelength-dependent and mode-dependent WTM processes. The direct measurement of the WTM of the spatial modes opens a convenient route to estimate group velocity dispersion, differential mode delay, and the effective refractive index of different spatial modes. This is applicable to TSI and ultrafast optical imaging, as well as broader areas such as telecommunications.

Fast-Gated 16 × 1 SPAD Array for Non-Line-of-Sight Imaging Applications

In this paper we present a novel single-photon detector specifically designed for Non-Line-Of-Sight (NLOS) imaging applications within the framework of the DARPA REVEAL program. The instrument is based on a linear 16 × 1 Complementary Metal-Oxide-Semiconductor (CMOS) Single-Photon Avalanche Diode (SPAD) array operated in fast-gated mode by a novel fast-gating Active Quenching Circuit (AQC) array, which enables the detectors with sub-ns transitions thanks to a SPAD-dummy approach. The detector exhibits a timing resolution better than 50 ps (Full Width at Half Maximum - FWHM) at a measurement repetition rate up to 40 MHz, and provides 16 independent outputs compatible with commercial Time-Correlated Single-Photon Counting (TCSPC) instrumentation. The instrument has been experimentally characterized and operated in preliminary NLOS imaging acquisitions where a 40 × 60 cm hidden object is successfully reconstructed by scanning over a grid of 150 × 150 positions.

Real-time multispectral fluorescence lifetime imaging using Single Photon Avalanche Diode arrays

Autofluorescence spectroscopy has emerged in recent years as a powerful tool to report label-free contrast between normal and diseased tissues, both in vivo and ex vivo. We report the development of an instrument employing Single Photon Avalanche Diode (SPAD) arrays to realize real-time multispectral autofluorescence lifetime imaging at a macroscopic scale using handheld single-point fibre optic probes, under bright background conditions. At the detection end, the fluorescence signal is passed through a transmission grating and both spectral and temporal information are encoded in the SPAD array. This configuration allows interrogation in the spectral range of interest in real time. Spatial information is provided by an external camera together with a guiding beam that provides a visual reference that is tracked in real-time. Through fast image processing and data analysis, fluorescence lifetime maps are augmented on white light images to provide feedback of the measurements in real-time. We validate and demonstrate the practicality of this technique in the reference fluorophores and in articular cartilage samples mimicking the degradation that occurs in osteoarthritis. Our results demonstrate that SPADs together with fibre probes can offer means to report autofluorescence spectral and lifetime contrast in real-time and thus are suitable candidates for in situ tissue diagnostics.