Real-Time Multistep Time-Series Prediction of Driver’s Head Pose During IVIS Secondary Tasks for Human–Machine Codriving and Distraction Warning Systems

Abstract

An accurate prediction of a driver’s head pose in the in-vehicle information system (IVIS) tasks is essential for both distraction warning and vehicle takeover rule-making in human–machine codriving. In this study, a real-world experiment is conducted to collect data on driver behavior and vehicle state during the IVIS tasks under different road conditions. An innovative scheme is proposed for multistep time-series prediction of a driver’s head pose. The proposed scheme consists of two main parts: a data-processing module and a model module. Furthermore, a combination of a machine-learning-based method and a geometric method is used to estimate a driver’s head pose, and the MediaPipe model is used to extract the hand data. In the model module, a convolutional neural network (CNN), which can automatically capture short-term local dependencies in the head pose data, and a bidirectional long short-term memory (BiLSTM), which can capture long-term macro dependencies, are combined. In addition, an attention mechanism is used to weigh important time periods and variables. Finally, the performance of the proposed hybrid CNN-BiLSTM-attention model in the head pose prediction is validated, and the proposed model is compared with three traditional machine-learning-based models and the CNN-BiLSTM model. The comparison results show that the proposed model outperforms the other models in the prediction steps and can provide reliable driver’s head pose data for distraction warning and takeover rules.