End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning

Abstract

Drivers’ decision and their corresponding behaviors are important aspects that can affect the driving safety, and it is necessary to understand the driver behaviors in real-time. In this study, an end-to-end driving-related tasks recognition system is proposed. Specifically, seven common driving activities are identified, which are normal driving, right mirror checking, rear mirror checking, left mirror checking, using in-vehicle video device, texting, and answering mobile phone. Among these, the first four activities are regarded as normal driving tasks, while the rest three are divided into distraction group. The images are collected using a consumer range camera, namely, Kinect. In total, five drivers are involved in the naturalistic data collection. Before training the identification model, the raw images are first segmented using a Gaussian mixture model (GMM) to extract the driver region from the background. Then, a pre-trained deep convolutional neural network (CNN) model is trained to classify the behaviors, which directly takes the processed RGB images as the input and outputs the identified label. In this work, the AIexNet is selected as the pre-trained CNN model. Then, to reduce the training cost, the transfer learning mechanism is applied to the CNN model. An average of 79\% detection accuracy is achieved for the seven driving tasks. The proposed integration model can be used as a low-cost driver distraction and dangerous tasks recognition modeL.