Direct Time-of-flight Research Articles

Path Tracing-Inspired Modeling of Non-Line-of-Sight SPAD Data.

Non-Line of Sight (NLOS) imaging has gained attention for its ability to detect and reconstruct objects beyond the direct line of sight, using scattered light, with applications in surveillance and autonomous navigation. This paper presents a versatile framework for modeling the temporal distribution of photon detections in direct Time of Flight (dToF) Lidar NLOS systems. Our approach accurately accounts for key factors such as material reflectivity, object distance, and occlusion by utilizing a proof-of-principle simulation realized with the Unreal Engine. By generating likelihood distributions for photon detections over time, we propose a mechanism for the simulation of NLOS imaging data, facilitating the optimization of NLOS systems and the development of novel reconstruction algorithms. The framework allows for the analysis of individual components of photon return distributions, yielding results consistent with prior experimental data and providing insights into the effects of extended surfaces and multi-path scattering. We introduce an optimized secondary scattering approach that captures critical multi-path information with reduced computational cost. This work provides a robust tool for the design and improvement of dToF SPAD Lidar-based NLOS imaging systems.

Multi-laser source interference suppression using the time-coding method for SPAD-based flash DToF systems

Flash-type direct time-of-flight (DToF) image sensors use an in-pixel successive approximation register time-to-digital converter (SAR TDC) for time quantization. However, in a scene where multiple DToF systems exist simultaneously, different laser signals from multiple sources will produce mutual signal interference between DToF systems, causing the DToF system’s incorrect measurement. In this paper, we present a method called time coding, which inserts delay time bins between different working periods to suppress the interference laser together with the SAR TDC. The time-coding method is designed using a 110 nm complementary metal oxide semiconductor (CMOS) technology and verified by behavioral model and circuit simulation. Regardless of traditional systems or systems equipped with time coding, DToF systems with certain patterns of time coding can reduce interference noise by at least 95%, maintaining a measurement accuracy of 99% or higher at long distances.

Depth Quality Improvement with a 607 MHz Time-Compressive Computational Pseudo-dToF CMOS Image Sensor

In this paper, we present a prototype pseudo-direct time-of-flight (ToF) CMOS image sensor, achieving high distance accuracy, precision, and robustness to multipath interference. An indirect ToF (iToF)-based image sensor, which enables high spatial resolution, is used to acquire temporal compressed signals in the charge domain. Whole received light waveforms, like those acquired with conventional direct ToF (dToF) image sensors, can be obtained after image reconstruction based on compressive sensing. Therefore, this method has the advantages of both dToF and iToF depth image sensors, such as high resolution, high accuracy, immunity to multipath interference, and the absence of motion artifacts. Additionally, two approaches to refine the depth resolution are explained: (1) the introduction of a sub-time window; and (2) oversampling in image reconstruction and quadratic fitting in the depth calculation. Experimental results show the separation of two reflections 40 cm apart under multipath interference conditions and a significant improvement in distance precision down to around 1 cm. Point cloud map videos demonstrate the improvements in depth resolution and accuracy. These results suggest that the proposed method could be a promising approach for virtually implementing dToF imaging suitable for challenging environments with multipath interference.

A low-power DTOF image sensor with double-frame-multiplexing mode for long-range detection

A novel time-to-digital converter (TDC) operation mode, called double-frame-multiplexing (DFM), is proposed for time-of-flight (TOF) measurements. It is based on the separate detection of the minimum distance and depth of the measured targets, which reduces the power consumption of in-pixel TDCs and expands the maximum detection distance under high resolution. A 64 × 64 direct TOF image sensor chip is fabricated in a 0.18 μm CMOS process. Driven by a 125 MHz multi-phase clock and a 50 MHz data readout clock, the chip achieves a maximum frame rate of 20 kframe/s, a maximum detection distance of 750 m (5 μs), and a maximum depth of field of 3 m (20 ns) with a temporal resolution of 1 ns. The DFM mode reduces the average power consumption to 60 mW without consideration of SPADs. The worst-case measured differential nonlinearity is −0.61/+0.69 least significant bit (LSB) and the maximum integral nonlinearity is −1.06/+0.66 LSB.

Plastic Classification Using Optical Parameter Features Measured with the TMF8801 Direct Time-of-Flight Depth Sensor.

We demonstrate a methodology for non-contact classification of five different plastic types using an inexpensive direct time-of-flight (ToF) sensor, the AMS TMF8801, designed for consumer electronics. The direct ToF sensor measures the time for a brief pulse of light to return from the material with the intensity change and spatial and temporal spread of the returned light conveying information on the optical properties of the material. We use measured ToF histogram data of all five plastics, captured at a range of sensor to material distances, to train a classifier that achieves 96% accuracy on a test dataset. To extend the generality and provide insight into the classification process, we fit the ToF histogram data to a physics-based model that differentiates between surface scattering and subsurface scattering. Three optical parameters of the ratio of direct to subsurface intensity, the object distance, and the time constant of the subsurface exponential decay are used as features for a classifier that achieves 88% accuracy. Additional measurements at a fixed distance of 22.5 cm showed perfect classification and revealed that Poisson noise is not the most significant source of variation when measurements are taken over a range of object distances. In total, this work proposes optical parameters for material classification that are robust over object distance and measurable by miniature direct time-of-flight sensors designed for installation in smartphones.

A 2-Stage EM Algorithm for Online Peak Detection, an Application to TCSPC Data

Time-Correlated Single Photon Counting (TCSPC) sensors based on the combination of Single Photon Avalanche Diodes (SPADs), Time to Digital Converters (TDCs) and histogram-builders have become a standard for depth imaging using direct Time of Flight (D-ToF). However, one of the main current issue of this working mode is intrinsically due to the large amount of data required to generate pixelwise ToF histograms. To overcome this limitation, a 2-stage Expectation Maximization (EM) algorithm for online peak detection with its Register Transfer Level hardware implementation is proposed. The online estimator consists in sequentially computing the statistical parameters that directly encode the latent spatial distance and intensity information of the illuminated scene. The evaluation on synthetic data demonstrates that with a compression ratio of 0.4%, our approach can efficiently reduce the pixel pitch by around 40% without sacrificing timing resolution.

Towards a Multi-Perspective Time of Flight Laser Ranging Device Based on Mirrors and Prisms

This paper investigates the feasibility of redirecting the field of view (FOV) of a light-based time-of-flight (ToF) ranging device, commonly known as a pulsed lidar, using fixed mirrors and prisms for possible future use in robotics. The emphasis is on configurations where the FOV redirection element is positioned beyond the ranging device’s dead zone. A custom made direct ToF ranging device with time-over-threshold (TOT)-based walk error compensation was used to evaluate the effects of the FOV redirecting optics on range measurement accuracy and precision. The tests include redirecting the FOV with a clean prism with anti-reflective (AR) coating on its legs, as well as with a regular and a first surface mirror in both a clean and dusted state. The study finds the prism to be unsuitable due to parasitic reflections, which ruin the ranging data. The clean mirrors were found to have no noticeable effect on ranging accuracy. When they are dusty, mirrors introduce a negative measurement error. This effect is the most pronounced when a mirror is positioned toward the end of the partial dead zone of the ToF rangefinder, but loses influence as the mirror is moved farther away. The error is attributed to the parasitic reflection off dust on the mirror, which reduces the time of detection of the pulse reflected off the real target, and interferes with the walk error compensation by widening the detected pulse.

A 16-Channel Analog CMOS SiPM With On-Chip Front-End for D-ToF LiDAR

This article presents a 16-channel analog silicon photomultiplier (SiPM) with on-chip front-end for direct time-of-flight (D-ToF) LiDAR applications. The proposed receiver is mainly composed of 16-channel SiPM, variable gain amplifier (VGA) and time-to-digital converter (TDC). Each SiPM channel consists of 256 microcells, and their outputs are connected to a common output terminal in parallel. A novel active quenching circuit is adopted to reduce the long exponential tail in conventional SiPM and enable the capability of multi-echo detection. Current steering circuits are adopted within microcells to make the output current of SiPM immune to SPAD gain variations. The receiver was fabricated in 180-nm HV CMOS technology and integrated into the 16-line LiDAR prototype with optical components. Measurement results show that the sensor is capable of 20 m range imaging with 3 cm accuracy under 40 klux background light conditions. With the mechanical scanning system, a high-resolution image ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$240\times16$ </tex-math></inline-formula> ) can be obtained.

Technical Note: Range verification of pulsed proton beams from fixed-field alternating gradient accelerator by means of time-of-flight measurement of ionoacoustic waves.

Ionoacoustics is one of the promising approaches to verify the beam range in proton therapy. However, the weakness of the wave signal remains a main hindrance to its application in clinics. Here we studied the potential use of a fixed-field alternating gradient accelerator (FFA), one of the accelerator candidates for future proton therapy. For such end, magnitude of the pressure wave and range accuracy achieved by the short-pulsed beam of FFA were assessed, using both simulation and experimental procedure. A 100MeV proton beam from the FFA was applied on a water phantom, through the acrylic wall. The beam range measured by the Bragg peak (BP)-ionization chamber (BPC) was 77.6mm, while the maximum dose at BP was estimated to be 0.35Gy/pulse. A hydrophone was placed 20mm downstream of the BP, and signals were amplified and stored by a digital oscilloscope, averaged, and low-pass filtered. Time-of-flight (TOF) and two relative TOF values were analyzed in order to determine the beam range. Furthermore, an acoustic wave transport simulation was conducted to estimate the amplitude of the pressure waves. The range calculated when using two relative TOF was 78.16±0.01 and 78.14±0.01mm, respectively, both values being coherent with the range measured by the BPC (the difference was 0.5-0.6mm). In contrast, utilizing the direct TOF resulted in a range error of 1.8mm. Fivefold and 50-fold averaging were required to suppress the range variation to below 1mm for TOF and relative TOF measures, respectively. The simulation suggested the magnitude of pressure wave at the detector exceeded 7 Pascal. A submillimeter range accuracy was attained with a pulsed beam of about 21ns from an FFA, at a clinical energy using relative TOF. To precisely quantify the range with a single TOF measurement, subsequent improvement in the measuring system is required.

Low-cost SPAD sensing for non-line-of-sight tracking, material classification and depth imaging

Time-correlated imaging is an emerging sensing modality that has been shown to enable promising application scenarios, including lidar ranging, fluorescence lifetime imaging, and even non-line-of-sight sensing. A leading technology for obtaining time-correlated light measurements are single-photon avalanche diodes (SPADs), which are extremely sensitive and capable of temporal resolution on the order of tens of picoseconds. However, the rare and expensive optical setups used by researchers have so far prohibited these novel sensing techniques from entering the mass market. Fortunately, SPADs also exist in a radically cheaper and more power-efficient version that has been widely deployed as proximity sensors in mobile devices for almost a decade. These commodity SPAD sensors can be obtained at a mere few cents per detector pixel. However, their inferior data quality and severe technical drawbacks compared to their high-end counterparts necessitate the use of additional optics and suitable processing algorithms. In this paper, we adopt an existing evaluation platform for commodity SPAD sensors, and modify it to unlock time-of-flight (ToF) histogramming and hence computational imaging. Based on this platform, we develop and demonstrate a family of hardware/software systems that, for the first time, implement applications that had so far been limited to significantly more advanced, higher-priced setups: direct ToF depth imaging, non-line-of-sight object tracking, and material classification.

Wide dynamic laser ranging based on diode laser and photon-counting techniques

Ultrafast diode laser and photon-counting techniques were united to solve the trade-off between the dynamic range and high precision in the time of flight (TOF) laser ranging. Based on the direct time of flight (D-TOF) laser ranging method, we developed a wide dynamic laser ranging system. The peak power of the emission laser can reach 70 W, the width of pulsed laser is less than 2 ns, and the repetition rate can reach 500 kHz. The experimental results show that the precision is 7.5 mm when the echo signal is saturated. The maximum range can reach kilometer level, the accuracy is within 30 cm, and the repetition rate of measurement can reach 100 kHz. The system has the advantages of small size, low cost, fast detection speed, and high accuracy, which can be used in the fields of dynamic target measurement and high-speed environment perception.

A Simple I-ToF Module Based on Avalanche Photodiode

This letter presents for the first time a new type of indirect time-of-flight (ToF) detection unit based on avalanche photodiode (APD) which is simple in structure and easy to integrate. Leveraging the advantages afforded by the direct ToF and indirect ToF technology, it has been verified by simulation that it has good time resolution, and the measurement results show a standard deviation of 3.17 mm at 4 m and less than 35 cm for distances up to 7.5 m. At the same time, this work proposes a new lateral APD model with a high responsivity reaching up to 4.37 A/W and low breakdown voltage of ~9.4 V.

A 4-GS/s 80-dB DR Current-Domain Analog Frontend for Phase-Coded Pulse-Compression Direct Time-of-Flight Automotive Lidar

A 4-GS/s current-domain analog front end (CurAFE) for phase-coded pulse-compression direct time-offlight (dTOF) automotive Lidar has been realized in 130-nm CMOS. The permissible eye-safe laser energy is transmitted in the form of a burst of optical sub-pulses. The burst embeds a phase coding designed to have an auto-correlation function with a maximum peak-to-sidelobe ratio. The optical reflection of this burst, reflected from a target, is then detected by an avalanche photodiode (APD). The APD's output current drives the CurAFE, whose 4-GS/s single-bit output is processed by a digital matched filter for pulse-compression and TOF estimation through cross correlation. The 200-MHz bandwidth frontend detects bursts with bitrates up to 200 Mb/s (5-ns sub-pulsewidth) and is designed for 80-dB DR without saturation and 5-pA/√Hz input-referred noise. A current-feedback dc servo loop at its input cancels the background-light-induced detector current. Compared to state-of-the-art single-pulse Lidars, operating with peak optical transmit powers ~50 W, this work achieves an inaccuracy of 0.225 m (1σ) over 60-m range at 10x lower peak transmit power of ~3.5 W.

Modeling and Analysis of a Direct Time-of-Flight Sensor Architecture for LiDAR Applications.

Direct time-of-flight (DTOF) is a prominent depth sensing method in light detection and ranging (LiDAR) applications. Single-photon avalanche diode (SPAD) arrays integrated in DTOF sensors have demonstrated excellent ranging and 3D imaging capabilities, making them promising candidates for LiDARs. However, high background noise due to solar exposure limits their performance and degrades the signal-to-background noise ratio (SBR). Noise-filtering techniques based on coincidence detection and time-gating have been implemented to mitigate this challenge but 3D imaging of a wide dynamic range scene is an ongoing issue. In this paper, we propose a coincidence-based DTOF sensor architecture to address the aforementioned challenges. The architecture is analyzed using a probabilistic model and simulation. A flash LiDAR setup is simulated with typical operating conditions of a wide angle field-of-view (FOV = 40) in a 50 klux ambient light assumption. Single-point ranging simulations are obtained for distances up to 150 m using the DTOF model. An activity-dependent coincidence is proposed as a way to improve imaging of wide dynamic range targets. An example scene with targets ranging between 8–60% reflectivity is used to simulate the proposed method. The model predicts that a single threshold cannot yield an accurate reconstruction and a higher (lower) reflective target requires a higher (lower) coincidence threshold. Further, a pixel-clustering scheme is introduced, capable of providing multiple simultaneous timing information as a means to enhance throughput and reduce timing uncertainty. Example scenes are reconstructed to distinguish up to 4 distinct target peaks simulated with a resolution of 500 ps. Alternatively, a time-gating mode is simulated where in the DTOF sensor performs target-selective ranging. Simulation results show reconstruction of a 10% reflective target at 20 m in the presence of a retro-reflective equivalent with a 60% reflectivity at 5 m within the same FOV.

A 30-frames/s, $252\times144$ SPAD Flash LiDAR With 1728 Dual-Clock 48.8-ps TDCs, and Pixel-Wise Integrated Histogramming

A 252 × 144 single-photon avalanche diode (SPAD) pixel sensor, called Ocelot, is reported for light detection and ranging (LiDAR). The sensor, fabricated in the 180-nm CMOS technology, features 1728 12-bit time-to-digital converters (TDCs) with 48.8-ps resolution (LSB). Each 126 pixels in a half-column are connected to six TDCs through a collision detection bus, which enables effective sharing of resources, and consequently a fill factor of 28% with a pixel pitch of 28.5 μm. The columnparallel TDCs, based on dual-clock architecture, exhibit a DNL of +0.48/-0.48 LSB and an INL of +0.89/-1.67 LSB; they are dynamically reallocated in a scalable daisy chain approach that enables a maximum of five photon detections per illumination cycle per half-column. The sensor can operate in timecorrelated single-photon counting (TCSPC) and single-photon counting (SPC) modes, while peak detection (PD) and partial histogramming (PH) are included in the operation of the sensor. The PD and PH modes are enabled by the first implementation of integrated histogramming for a full array via 3.32-Mb SRAMbased PH readout (PHR) scheme providing a 14.9-to-1 compression. Telemetry measurements up to 50 m achieve an accuracy of 8.8 cm and worst-case precision of 1.4 mm (σ). A flash LiDAR using direct time of flight (dTOF) and based on Ocelot is demonstrated, achieving depth imaging at short distances with a frame rate of 30 frames/s, employing an ultra-low power laser with an average power of 2 mW and peak power of 0.5 W.

Real-time computational photon-counting LiDAR

The availability of compact, low-cost, and high-speed MEMS-based spatial light modulators has generated widespread interest in alternative sampling strategies for imaging systems utilizing single-pixel detectors. The development of compressed sensing schemes for real-time computational imaging may have promising commercial applications for high-performance detectors, where the availability of focal plane arrays is expensive or otherwise limited. We discuss the research and development of a prototype light detection and ranging (LiDAR) system via direct time of flight, which utilizes a single high-sensitivity photon-counting detector and fast-timing electronics to recover millimeter accuracy three-dimensional images in real time. The development of low-cost real time computational LiDAR systems could have importance for applications in security, defense, and autonomous vehicles.

Solution Based on Dynamical Approach for Multiple-Revolution Lambert Problem

T HE classical Lambert problem deals with determination of the transfer velocity vector at an initial position to reach a final position in the specified time of flight. The multiple-revolution Lambert problem (MRLP) results when the interception takes place after completing one or more revolutions around the central body. In the case of the classical Lambert problem, the angular displacement between two positions is between 0 and 2 , while for the MRLP, it exceeds 2 . The total time of flight for theMRLPwithN revolutions is the sum ofN times the period of transfer orbit and the direct time of flight. There exists a total of 2N 1 solutions for N complete revolutions.A simple algorithm to solve theMRLP is of great interest for spacecraft interception, rendezvous, and interplanetary transfer. The MRLP approach allows a larger navigational flexibility and results in lower energy orbits than the direct transfer arcs. A good amount of research has been made on the classical Lambert problem. Some of the approaches are extended for the MRLP solution as well as optimization for fuel or velocity. The solution for the classical Lambert problem was first attempted by Gauss [1]. The universal variable formulation method was applied for solving the classical Lambert problem as well as the MRLP in Bate et al. [2]. Battin and Vaughan [3] attempted the classical Lambert problem with the semimajor axis as an independent parameter and solved it with the help of the hypergeometric series. Separate algorithms for the high and low energy transfers to the MRLP with the Battin and Vaughan formulation were evolved by Loechler [4]. Prussing [5] used the Lagrange formulation for the minimum-fuel multiple-revolution algorithm for an elliptic orbit to determine all the possible transfer trajectories. Shen and Tsiotras [6] extended it for the minimum velocity fixed-time two-impulse transfer algorithm between two coplanar circular orbits. The transverse eccentricity vector-based algorithm was given by Avanzini [7]. He et al. [8] extended Avanzini’s [7] approach for the MRLP and improved the convergence of the algorithm. All these approaches used the Kepler time equation for calculating the time of flight. A dynamical approach has been developed by Nelson and Zarchan [9] for solving the classical Lambert problem. It is based on the closedform solution of the equations of motion given by Wheelon [10]. Velocity and time of flight are obtained as a function of flight-path angle as an independent parameter. The iterating variable (flight-path angle) and the terminating condition (time of flight) are physical parameters of the transfer orbit [9]. In this Note, we report a novel algorithm based on the dynamical approach for solving the MRLP with the flight-path angle as an independent parameter. The expression for the total time of flight is obtained. The high and low energy orbits get separated using a derivative of total time of flight with respect to the flight-path angle. The algorithm is fast and easy to understand. It is noteworthy that the results obtained are in complete agreement with He et al. [8].

Very High Energy Neutron Scattering from Hydrogen

The neutron scattering from hydrogen in polythene has been measured with the direct time-of flight spectrometer, MARI, at the ISIS facility of the Rutherford Appleton Laboratory with incident neutron energies between 0.5 eV and 600 eV. The results of experiments using the spectrometer, VESUVIO, have given intensities from hydrogen containing materials that were about 60% of the intensity expected from hydrogen. Since VESUVIO is the only instrument in the world that routinely operates with incident neutron energies in the eV range we have chosen to measure the scattering from hydrogen at high incident neutron energies with a different type of instrument. The MARI, direct time-of-flight, instrument was chosen for the experiment and we have studied the scattering for several different incident neutron energies. We have learnt how to subtract the gamma ray background, how to calibrate the incident energy and how to convert the spectra to an energy plot . The intensity of the hydrogen scattering was independent of the scattering angle for scattering angles from about 5 degrees up to 70 degrees for at least 3 different incident neutron energies between 20 eV and 100 eV. When the data was put on an absolute scale, by measuring the scattering from 5 metal foils with known thicknesses under the same conditions we found that the absolute intensity of the scattering from the hydrogen was in agreement with that expected to an accuracy of ± 5.0% over a wide range of wave-vector transfers between 1 and 250 Å−1. These measurements show that it is possible to measure the neutron scattering with incident neutron energies up to at least 100 eV with a direct geometry time-of-flight spectrometer and that the results are in agreement with conventional scattering theory.

Normalisation of the resolution function in TOF technique

The normalisation factors of the resolution function are the basic components for the conversion of measured neutron scattering spectra to the scattering function The instrumental resolution function and its normalisation factors are derived independently of any sample and its As shown earlier for Three-Axis Spectrometer, these factors, also in time of flight (TOF) technique, are symmetric for the incoming beam, with and the scattered beam, with Here k I and k F are the wave vectors of the incoming and scattered neutrons; describes the spectrum of the source in phase-space and the detector efficiency. In direct TOF the conversion factor is In inverse TOF it is The latter depends on the neutron source. For thermal neutrons (gas in thermal equilibrium) the conversion factor is where the wave vector k T corresponds to temperature. For epithermal neutrons, in the eV range, the conversion factor is constant.