Advanced time-of-flight range camera with novel real-time 3D image processing

Abstract

We present a solid state range camera covering measuring distances from 2 m to 25 m and novel real-time 3D image processing algorithms for object detection, tracking and classification based on the three-dimensional features of the camera's output data. The technology is based on a 64x8 pixel array CMOS image sensor which is capable of capturing three-dimensional images by executing indirect time-of-flight (ToF) measurement of NIR laser pulses emitted by the camera and reflected by the objects in the cameras field of view. Here the so-called "multiple double short time integration" (MDSI) method enables unprecedented reliability and robustness with respect to suppression of background irradiance and insensitiveness to reflectivity variations in the object scene. Output data are conventional intensity values and distance values with accuracies in the centimeter range at image repetition rates up to 100 Hz. An evaluation of the camera's performance in typical road safety related test scenarios is subject of this paper. Furthermore we introduce real-time image processing of the output data stream of the camera aiming at the segmentation of objects being located in the camera's surrounding and the derivation of reliable position, speed and acceleration estimates. The segmentation algorithm utilizes the position information of all three spatial dimensions as well as the intensity values and thus yields significant segmentation improvement compared to segmentation in conventional 2D pictures. Position, velocity and acceleration values of the segmented objects are estimated by means of Kalman filtering in 3D space. The filter is dynamically adapting to the measurement conditions to take care of changes of the scene data properties. Flow and performance of the whole processing chain are presented by means of example scenes.