Neuromorphic Vision Based Multivehicle Detection and Tracking for Intelligent Transportation System

Guang Chen,Jörg Conradt,Xingbo Liu,Kai Chen,Muhammad Aafaque,Florian Röhrbein,Alois Knoll,Jieneng Chen,Zhenshan Bing,Hu Cao,Walter Stechele,Gereon Hinz,Canbo Ye

doi:10.1155/2018/4815383

Abstract

Neuromorphic vision sensor is a new passive sensing modality and a frameless sensor with a number of advantages over traditional cameras. Instead of wastefully sending entire images at fixed frame rate, neuromorphic vision sensor only transmits the local pixel-level changes caused by the movement in a scene at the time they occur. This results in advantageous characteristics, in terms of low energy consumption, high dynamic range, sparse event stream, and low response latency, which can be very useful in intelligent perception systems for modern intelligent transportation system (ITS) that requires efficient wireless data communication and low power embedded computing resources. In this paper, we propose the first neuromorphic vision based multivehicle detection and tracking system in ITS. The performance of the system is evaluated with a dataset recorded by a neuromorphic vision sensor mounted on a highway bridge. We performed a preliminary multivehicle tracking-by-clustering study using three classical clustering approaches and four tracking approaches. Our experiment results indicate that, by making full use of the low latency and sparse event stream, we could easily integrate an online tracking-by-clustering system running at a high frame rate, which far exceeds the real-time capabilities of traditional frame-based cameras. If the accuracy is prioritized, the tracking task can also be performed robustly at a relatively high rate with different combinations of algorithms. We also provide our dataset and evaluation approaches serving as the first neuromorphic benchmark in ITS and hopefully can motivate further research on neuromorphic vision sensors for ITS solutions.

Highlights

Neuromorphic vision sensors, inspired by biological vision, use an event-driven frameless approach to capture transients in visual scenes
The algorithm models the respective collection of targets and measurements as random finite sets and applies the probability hypothesis density (PHD) recursively for posterior intensity propagation, which is basically the first order-statistic of the random finite set in time
We follow the current evaluation protocols for visual object detection and multiobject tracking. These protocols are designed for frame-based vision sensors, they are still suitable for quantitative evaluation of our tracking method

Summary

Introduction

Neuromorphic vision sensors, inspired by biological vision, use an event-driven frameless approach to capture transients in visual scenes. Apart from the low latency and high storage efficiency, neuromorphic vision sensors enjoy a high dynamic range of 120 dB In combination, these properties of neuromorphic vision sensors inspire entirely new designs of intelligent transportation systems. Frame-based vision sensors serve as the main information sources for vision perception tasks of ITS, which results in well-known challenges, such as the limited real-time performance and substantial computational costs. Traditional vision sensors waste memory access, energy, computational power, and time on the one hand and discard significant information between continuous frames on the other hand These properties bring about great limitations on its applications. A novel approach for the tracking system of the intelligent transportation systems (ITS) is proposed based on the neuromorphic vision sensor.

Related Work

Neuromorphic Vision Sensor and Dataset

Online Multitarget Detection and Tracking

Experiments and Results

Conclusion and Discussion