Spike Coding for Dynamic Vision Sensor in Intelligent Driving

Abstract

Dynamic vision sensor (DVS) as a bio-inspired camera, has shown great advantages in wide dynamic range and high temporal resolution imaging in contrast to conventional frame-based cameras. Its ability to capture high speed moving objects enables fast and accurate detection which plays a significant role in the emerging intelligent driving applications. The pixels in DVS independently respond to the luminance changes with output spikes. Thus, the spike stream conveying the ${x}$ -, ${y}$ -addresses, the firing time, and the polarity (ON/OFF), is quite different from conventional video frames. How to compress this kind of new data for efficient transmission and storage remains a big challenge, especially for on-board detection, monitoring and recording. To address this challenge, this paper first analyzes the spike firing mechanism and the spatiotemporal characteristics of the spike data, then introduces a cube-based spike coding framework for DVS. In the framework, an octree-based structure is proposed to adaptively partition the spike stream into coding cubes in both spatial and temporal dimensions, then several prediction modes are designed to exploit the spatial and temporal characteristics of spikes for compression, including address-prior mode and time-prior mode. To explore more flexibility, the intercube prediction is discussed extensively involving motion estimation and motion compensation. Finally, the experimental results demonstrate that our approach achieves an impressive coding performance with the average compression ratio of 2.6536 against the raw spike data, which is much higher than the results of conventional lossless coding algorithms.