Driving maneuver early detection via sequence learning from vehicle signals and video images

Abstract

Driving Maneuver Early Detection (DMED) is particularly useful for many applications of intelligent vehicle systems, including driver warning and collision avoidance systems. In this paper, we introduce a robust DMED model, denoted as University of Michigan Dearborn (UMD)-DMED, developed using innovative features and deep learning techniques. The UMD-DMED model contains three major computational components, distance based representation of driving context, combined vehicle trajectory features and visual features, and a Long Short-Term Memory (LSTM)-based neural network that captures temporal dependencies of driving maneuvers. To properly evaluate the performances of UMD-DMED, we developed two DMED systems based on the UMD-DMED model, one system is based on partially observed evidence of maneuver events, and another on features observed ahead of the time that driving maneuvers take place. We conducted the extensive experiments using a data set containing 1078 maneuver events extracted from 37 hours of real world driving trips. The results demonstrate that the UMD-DMED model is capable of learning the latent features of five different classes of driving maneuvers, i.e. left turn, right turn, left lane change, right lane change, driving straight. Comparing to four different state-of-the-art DMED systems, the UMD-DMED achieved better detection performances in both, the detection based on partial observations of driver maneuvering, and based on driving context observed ahead-of-time.