Detecting Driver's Braking Intention Using Recurrent Convolutional Neural Networks Based EEG Analysis

Abstract

Driving assistance system has been recently studied to prevent emergency braking situations by combining external information on radar or camera devices and internal information on driver's intention. Electroencephalography (EEG) is an effective method to read user's intention with high temporal resolution. Our proposed system is mainly contributed to detecting driver's braking intention prior to stepping on the brake pedal in the emergency situation. We investigated early event-related potential (ERP) curves evoked by visual sensory process in emergency situation by using recurrent convolutional neural networks (RCNN) model. RCNN model has advantages to capture contextual and spatial patterns of brain signal. RCNN model is composed of a convolutional layer, two recurrent convolutional layers (RCLs), and a softmax layer. Fourteen participants drove for 120 minutes with two types of emergency situations and a normal driving situation in a virtual driving environment. In this article, early ERP showed a potential to be used for classifying the driver's braking intention. The classification performances based on RCNN and regularized linear discriminant analysis (RLDA) at 200 ms post-stimulus time were 0.86 AUC score and 0.61 AUC score respectively. Following the results, braking intention was recognized at 380 ms earlier based on early ERP patterns using RCNN model than the brake pedal. Our system could be applied to other brain-computer interface (BCI) system for minimizing detection time by capturing early ERP curves based on RCNN model.