Real-Time Driver Behavior Detection Based on Deep Deformable Inverted Residual Network With an Attention Mechanism for Human-Vehicle Co-Driving System

Abstract

Real-time driver behavior detection (DBD) is essential for developing driver-centered human-vehicle co-driving systems. This paper proposes an accurate and easily implementable DBD method based on driver behavior images and deep learning. Convolutional neural networks have difficulty dealing with changes in driver behavior images (rotation, scale, and translation). They suffer from a tradeoff between accuracy and number of trainable parameters, noise interference in images of driver behavior, and the black-box problem of neural networks, limiting the effectiveness of current DBD methods. Therefore, we design a novel deep learning-based DBD method consisting of a deep deformable inverted residual network with an attention mechanism. Deformable convolution is used to deal with image rotation and translation. An inverted residual block and linear bottleneck approach based on depthwise separable convolution is used to reduce the number of trainable parameters while maintaining high accuracy. An attention mechanism based on soft thresholding is incorporated into the nonlinear transformation layers to extract driver behavior-relevant features. We also innovatively propose a visualization method to improve the interpretability of the proposed method. The experimental results show that the proposed method can accurately detect driver behavior (95.17% mAP on the Kaggle driving test dataset), significantly outperforming state-of-the-art methods regarding accuracy, real-time performance, and reliability.