Driving Behavior Recognition Research Articles

On-Board Unit (OBU)-Supported Longitudinal Driving Behavior Monitoring Using Machine Learning Approaches.

Driving behavior recognition can provide an important reference for the intelligent vehicle industry and probe vehicle-based traffic estimation. The identification of driving behavior using mobile sensing techniques such as smartphone- and vehicle-mounted terminals has gained significant attention in recent years. The present work proposed the monitoring of longitudinal driving behavior using a machine learning approach with the support of an on-board unit (OBU). Specifically, based on velocity, three-axis acceleration and three-axis angular velocity data were collected by a mobile vehicle terminal OBU; through the process of data preprocessing and feature extraction, seven machine learning algorithms, including support vector machine (SVM), random forest (RF), k-nearest neighbor algorithm (KNN), logistic regression (LR), BP neural network (BPNN), decision tree (DT), and the Naive Bayes (NB), were applied to implement the classification and monitoring of the longitudinal driving behavior of probe vehicles. The results show that the three classifiers SVM, RF and DT achieved good performances in identifying different longitudinal driving behaviors. The outcome of the present work could contribute to the fields of traffic management and traffic safety, providing important support for the realization of intelligent transport systems and the improvement of driving safety.

Driving Behaviors Recognition Using Deep Neural Networks

Road accidents are skyrocketing, and traffic safety is a severe problem around the world. Many road traffic deaths are related to drivers’ unsafe behaviors. In this paper, we propose two different deep-learning models which classify the driver’s actions in a 60-second time frame into two main categories: Normal and Aggressive driving based on GPS data collected at 1 Hz, which is later preprocessed and passed to the proposed models to identify dominant driving behavior in each time frame. The models achieved an accuracy of 93.75 percent in real-world tests, which proves the efficiency of this method in driving behavior recognition.

Smart Driver Behavior Recognition and 360-Degree Surround-View Camera for Electric Buses

Smart Driver Behavior Recognition and 360-Degree Surround-View Camera for Electric Buses

Driver Behavior Recognition Based on Neural Networks Theory

The article deals with the problem of driver behavior while driving the vehicle. Driver distraction can lead to serious accidents that threaten human life and public property around the world. Solving the problem of preventing dangerous driving behavior will reduce the risk of getting into an accident in the future. Thus, there is a need for a smart vehicle that will support driver behavior recognition functionality. A possible solution to the problem using an artificial neural network for automatic recognition of driver behavior on a real set of driver behavior data is considered. The high accuracy and efficiency of the developed model recognition is obtained.

A review on machine learning-based models for lane-changing behavior prediction and recognition

A major aspect in the development of advanced driving assistance systems (ADASs) is the research in developing human driving behavior prediction and recognition models. Recent contributions focus on developing these models for estimating different driving behaviors like lane or speed change. Thus, the models are incorporated into the ADAS to generate warnings and hints for safe maneuvers. Driving behavior recognition and prediction models are generally developed based on machine learning (ML) algorithms and are proven to generate accurate estimations. Previous review research contributions tend to focus on ML-based models for the prediction and recognition of speed change, trajectory change, and even driving styles. Due to high number of driving errors occurring during a lane change, a state-of-art review of different ML-based models for lane-changing behavior prediction and recognition is helpful to present a comparison between different models in terms of structure, influencing input variables, and performance. This enables the integration of the most efficient model for the development of ADASs to avoid accidents during a lane change. First, definitions and terms related to the model’s task and evaluation metrics used to evaluate the model’s performance are described to improve the readability. Then, the different input variables of the models affecting the lane-changing behaviors are presented. Next, a review of the models developed based on well-known approaches, such as artificial neural network (ANN), hidden Markov model (HMM), and support vector machine (SVM), using different input variables is given. Three lane-changing behaviors are focused on here: left/right lane change and lane keeping. The advantages and disadvantages of the different ML models with a comparison are summarized as well. Finally, the improvements required in the future are discussed.

Sparse least squares support vector machine based methods for vehicle driving behavior recognition

Intelligent vehicles are expected to accurately recognize the driving intentions of surrounding vehicles so as to precisely identify the hazards to automatic driving and accomplish reasonable motion planning. This paper introduces a model for vehicles driving behavior recognition (VDBR) based on Sparse Least Squares Support Vector Machine (S-LSSVM) by means of machine learning methods, with the subject vehicle and its surrounding vehicles as the research subject. First, the relative lateral displacement and relative lateral speed between vehicles are captured as the eigenvectors after calculation of trajectory curvature and change time window. Then, the pruning algorithm is used to make Least Squares Support Vector Machine (LSSVM) training samples sparse and the Particle Swarm Optimization algorithm (PSO) is employed to accomplish parameter tuning of S-LSSVM. Thus, a modified S-LSSVM model is constructed to grasp the interaction behavior between vehicles. The experiment results demonstrate that the S-LSSVM based model obtain better accuracy and timeliness compared with SVM and LSSVM on the Next Generation Simulation (NGSIM) dataset and the data from autonomous driving experimental platform.

Driver Behavior Modeling Toward Autonomous Vehicles: Comprehensive Review

Driver behavior models have been used as input to self-coaching, accident prevention studies, and developing driver-assisting systems. In recent years, driver behavior recognition has revolutionized autonomous vehicles (AVs) and traffic management studies. This comprehensive survey provides an up-to-date review of the different driver behavior models and modeling approaches. In heterogeneous streets where humans and autonomous vehicles operate simultaneously, predicting the intent and action of human drivers is crucial for AVs with the help of wireless communication and artificial intelligence (AI) technologies. Therefore, the review also summarizes the applications of driver behavior modeling (DBM) for effective behavior recognition and human-like AV driving. Moreover, the review also covers the application of DBM in capturing behaviors of complex dynamic driving tasks. In this review, we solely cover car-following (CF) and lane-changing (LC) maneuvers.

Distracted driving behaviour recognition based on transfer learning and model fusion

Distracted driving behaviour recognition based on transfer learning and model fusion

Distracted driving behaviour recognition based on transfer learning and model fusion

In the recognition of distracted driving behaviour, traditional manual feature extraction is subjective and complex; single deep convolutional network also has problems such as insufficient generalisation performance and stability. To solve the above problems, this paper proposes a distracted driving behaviour recognition method based on transfer learning and model fusion. First, based on the transfer learning method, the deep convolutional neural network models ResNet18 and ResNet34 are used to extract the features of some images, respectively. Furthermore, the pre-trained model is fine-tuned to obtain four deep convolutional neural network models. Finally, the four network models are fused by stacking method, using 5-fold cross-validation method to reduce over-fitting. Experimental results show that the recognition accuracy of distracted driving behaviour after model fusion reaches 95.47%. The fusion model has higher model generalisation performance and recognition accuracy, which can provide certain technical support for the research of distracted driving behaviour recognition.

BiRSwinT: Bilinear full-scale residual swin-transformer for fine-grained driver behavior recognition

Driver behavior recognition has attracted extensive attention recently. Numerous methods have been developed on the basis of various deep neural networks. However, the existing models still suffer from various challenges in solving the downstream fine-grained recognition task, because many driver behaviors share similar global shape / appearances, and can only be discriminated by subtle but meaningful actions of local body parts. To remedy this emerging issue of fine-grained driver behavior recognition, we present a bilinear full-scale residual Swin-Transformer network (BiRSwinT) to learn and fuse the global shape/appearance and the local discriminative cues of driver actions. Specifically, a transformer-based dual-stream structure is designed, which contains two parallel branches: a global representation branch and a local residual branch. To learn the multi-scale local cues implied in driver actions, a set of full-scale skip connections are introduced in the local residual branch. Then a bilinear fusion with multi-step training strategy is employed to leverage and combine the global representation and local detail features of driver actions. The proposed method has been validated on the AUC V1 and V2 datasets, and achieved the average Top-1 accuracy of 93.235% and 92.253%, respectively. Comparison experiments show that the proposed method has superior performance when compared to the state-of-the-art methods. The generalization experiments on the State-Farm dataset also demonstrate the superiority of the proposed framework and strategy. The source code is available at https://github.com/shadow2469/BiRSwinT.git.

Fatigue Driving Recognition Method Based on Multi-Scale Facial Landmark Detector

Fatigue driving behavior recognition in all-weather real driving environments is a challenging task. Accurate recognition of fatigue driving behavior is helpful to improve traffic safety. The facial landmark detector is crucial to fatigue driving recognition. However, existing facial landmark detectors are mainly aimed at stable front face color images instead of side face gray images, which is difficult to adapt to the fatigue driving behavior recognition in real dynamic scenes. To maximize the driver’s facial feature information and temporal characteristics, a fatigue driving behavior recognition method based on a multi-scale facial landmark detector (MSFLD) is proposed. First, a spatial pyramid pooling and multi-scale feature output (SPP-MSFO) detection model is built to obtain a face region image. The MSFLD is a lightweight facial landmark detector, which is composed of convolution layers, inverted bottleneck blocks, and multi-scale full connection layers to achieve accurate detection of 23 key points on the face. Second, the aspect ratios of the left eye, right eye and mouth are calculated in accordance with the coordinates of the key points to form a fatigue parameter matrix. Finally, the combination of adaptive threshold and statistical threshold is used to avoid misjudgment of fatigue driving recognition. The adaptive threshold is dynamic, which solves the problem of the difference in the aspect ratio of the eyes and mouths of different drivers. The statistical threshold is a supplement to solve the problem of driver’s low eye threshold and high mouth threshold. The proposed methods are evaluated on the Hunan University Fatigue Detection (HNUFDD) dataset. The proposed MSFLD achieves a normalized mean error value of 5.4518%, and the accuracy of the fatigue driving recognition method based on MSFLD achieves 99.1329%, which outperforms that of state-of-the-art methods.

BRAM-ED: Vehicle Trajectory Prediction Considering the Change of Driving Behavior

Trajectory prediction plays a key role in the decision-making system of autonomous vehicles. The existing trajectory prediction models have the problem of accuracy deterioration when driving behavior change. To solve this problem, a novel BRAM-ED trajectory prediction framework is proposed, which mainly consists of driving behavior recognition model, behavior attention mechanism (BAM) trajectory encoder, and behavior adaptive future trajectory decoder. First, a Bi-GLSTM network is designed to recognize real-time driving behavior. The graph structure is used to describe the complex dependencies among vehicles and recurrent cell is to capture temporal correlation. Subsequently, in contrast to traditional models, directly concatenate historical trajectories and interactive information as the input of prediction model, BAM is proposed to integrated trajectory and interaction features in this article. The BAM is designed to generate attention weights according to the change of driving behavior, and guide trajectory decoder to predict the future trajectories. Then, the proposed model is trained and validated on various public datasets, including HighD and NGSIM. Compared with existing optimal models, our prediction error has been reduced 44.66% at most. Furthermore, a typical cut-in scenario is designed based on hardware-in-loop platform. The experiment results show that the proposed model could recognize the change of driving behavior timely, and predict accurate trajectory.

Vehicles driving behavior recognition based on transfer learning

Due to the complexity of experiments to test driving behaviors and the high cost of data collection for some types of vehicles, e.g., heavy-duty freight vehicles, it is normally hard to develop a model with a small size of samples for higher performance to correctly recognize driving behavior patterns. This paper proposes an effective recognition method based on the Convolutional Neural Network (CNN) and transfer learning. Firstly, a CNN model was constructed that was coupled with multi-source data fusion, natural driving GPS data, and drivers’ facial expression data of online car-hailing, to recognize the feature maps of five driving behavior patterns, including acceleration, deceleration, turning, lane changing, and lane keeping. Secondly, the transfer learning algorithm was employed to fine-tune the pre-trained CNN model parameters with few natural driving data samples of heavy-duty freight vehicles, where data collection is traditionally very difficult. The experiments showed that this transferred model yields a higher performance with an accuracy score of 0.80 than the non-transferred one with an accuracy of 0.64 only. Additionally, such a transferred model converged very fast with a lower training cost. After only 1,000 training epochs, its performance is much better than that of the non-transferred model after 5,000 epochs. The results demonstrated that transfer learning is an effective potential method for driving behavior recognition and other similar studies where the sample size is relatively small due to various reasons.

Recognition of Trip-Based Aggressive Driving: A System Integrated With Gaussian Mixture Model Structured of Factor-Analysis, and Hierarchical Clustering

Recognition of aggressive driving behavior helps future research and practices in Intelligent Transport Systems. This paper tries to briefly explain the concepts related to aggressive driving behavior and introduces a system integrated with a novel machine learning algorithm for the recognition of trip-based aggressive driving behavior. The algorithm is a Gaussian Mixture Model (GMM) structured with Factor Analysis (FA), and Hierarchical Clustering (HC): common factors were extracted using FA, which is further applied to HC and GMM in the recognition of trip-based aggressive driving. The system is applied in a case study using data from the Shanghai Naturalistic Driving Study, for simulating data collection using the Advanced Driving Assistance System (ADAS) system in a real-traffic situation. Three behavior types (cautious, regular, and aggressive driving) were successfully clustered. For validity, the real aggressive driving behavior records were extracted based on the video, and the proposed system was compared with existing recognition methods. Results indicate that the accuracy of aggressive driving recognition of the system is higher than others (accuracy = 87%). This paper provides a reference in defining and determining aggressive driving, and a robust system for aggressive driving behavior recognition along with the trained algorithm, which can be used in real-world applications for improving driving safety with the applications in ADAS systems, auto-insurance industry.

A Review for the Driving Behavior Recognition Methods Based on Vehicle Multisensor Information

The frequent traffic accidents lead to a large number of casualties and large related financial losses every year; this serious state is owed to several factors; among those, driving behavior is one of the most imperative subjects to discuss. Driving behaviors mainly include behavior characteristics such as car-following, lane change, and risky driving behavior such as distraction, fatigue, or aggressive driving, which are of great help to various tasks in traffic engineering. An accurate and reliable method of driving behavior recognition is of great significance and guidance for vehicle driving safety. In this paper, the vehicle multisensor information, vehicle CAN bus data acquisition system, and typical feature extraction methods are summarized at first. And then, several driving behavior recognition models based on machine learning and deep learning are reviewed. Through a detailed analysis of the features of random forests, support vector machines, convolutional neural networks, and recurrent neural networks used to build driving behavior recognition models, the following findings are obtained: the driving behavior model constructed by traditional machine learning model is relatively mature but it is greatly affected by feature extraction, data scale, and model structure, which affects the accuracy of the final driving behavior recognition. Deep learning model based on a neural network has achieved high accuracy in identifying driving behavior, and it may gradually become the mainstream of constructing the driving behavior model with the development of big data, artificial intelligence technology, and computer hardware. Finally, this paper points out some content that needs to be further explored, to provide reference and inspiration for scholars in this field to continue to study the driving behavior recognition model in depth.

Dangerous Driving Behavior Recognition Based on Hand Trajectory

Dangerous driving behaviors in the process of driving will produce road traffic safety hazards, and even cause traffic accidents. Common dangerous driving behavior includes: eating, smoking, fetching items, using a handheld phone, and touching a control monitor. In order to accurately identify the dangerous driving behaviors, this study first uses the hand trajectory data to construct the dangerous driving behavior recognition model based on the dynamic time warping algorithm (DTW) and the longest common sub-sequence algorithm (LCS). Secondly, 45 subjects’ hand trajectory data were obtained by driving simulation test, and 30 subjects’ hand trajectory data were used to determine the dangerous driving behavior label. The matching degree of hand trajectory data of 15 subjects was calculated based on the dangerous driving behavior recognition model, and the threshold of dangerous driving behavior recognition was determined according to the calculation results. Finally, the dangerous driving behavior recognition algorithm and neural network algorithm are compared and analyzed. The dangerous driving behavior recognition algorithm has a fast calculation speed, small memory consumption, and simple program structure. The research results can be applied to dangerous driving behavior recognition and driving distraction warning based on wrist wearable devices.

Driving State Discrimination Algorithm Based on Lightweight Network and Contrast Learning

Driver misbehavior is one of the major traffic safety hazards as car ownership increases year by year. So, it is important to have driver fatigue detection and behavior recognition. Initially, given the fatigue detection problem that the images captured by the visible light camera cannot capture the eyes of a driver wearing sunglasses or eye glasses. As a solution, this paper introduces a DCT-HSV preprocessing algorithm for infrared images, which is believed to enhance the target characteristics of infrared images. The paper also introduces a more efficient lightweight SSD detection model, which achieves a better balance in terms of model size and detection performance. It also shows a better performance in self-built datasets and basic vehicle operation datasets. Secondly, aiming at the problems of high complexity and poor accuracy of the existing driving behavior detection model, this paper designs a driver abnormal behavior discrimination model based on the comparison twin, which has good performance on the Kaggle public dataset. The relevant experimental detection results show that the method constructed in this paper has high detection accuracy, low warning delay, and good practical use value.

Federated learning‐based trajectory prediction model with privacy preserving for intelligent vehicle

The existing trajectory prediction is mainly for specific road sections, which is poor to adapt to complex and changing traffic scenarios. Meanwhile, decentralized trajectory data is hard to be fully utilized in the data silo environment. To solve data silos in the intelligent vehicle industry, introducing federated learning methods in vehicular edge computing has attracted extensive attention. But traditional federated learning still has the potential to suffer from the mining of training data in the case of model privacy leakage. In this paper, a vehicle trajectory prediction method based on federated learning and homomorphic encryption has been presented, which adopts a three-layer architecture with a vehicle cluster, edge computing server, and cloud core network. Compared with traditional centralized deep learning methods, this approach enables joint modeling of multiple parties to improve the model's generalization performance. We used a proxy re-encryption algorithm to implement key distribution and also designed an encrypted federated network algorithm FAHEFL, which uses FAHE1 homomorphic encryption to protect the privacy of the model in parameter transmission. Each local model includes a driving behavior recognition module and trajectory output module. The driving behavior recognition module uses a 1D convolutional neural network to recognize the driving behavior, then input recognition results and historical trajectories to the trajectory output module, which uses LSTM neural network to output predicted trajectories. The experiment results show that the model built with FAHEFL has no more 1% error in the driving behavior recognition module than concentrated learning, while the minimum mean square error of the trajectory prediction module increased by only 2.5%. This article also discusses the performance between FAHEFL and well-known cryptographic federation network algorithms.

Bidirectional Posture-Appearance Interaction Network for Driver Behavior Recognition

Driver behavior recognition has become one of the most important tasks for intelligent vehicles. This task, however, is very challenging since the background contents in real-world driving scenarios are often very complex. More critically, the difference between driving behaviors is often very minor, making it extremely difficult to distinguish them. Existing methods often rely only on RGB frames (or skeleton data), which may fail to capture the minor differences between behaviors and appearance information of objects simultaneously and thus fail to achieve promising performance. To address the above issues, in this paper, we propose a bidirectional posture-appearance interaction network (BPAI-Net), which simultaneously considers RGB frames and skeleton ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , posture) data for driver behavior recognition. Specifically, we propose a posture-guided convolutional neural network (PG-CNN) and an appearance-guided graph convolutional network (AG-GCN) to extract appearance and posture features, respectively. To exploit the complementary information between appearance and posture, we use the appearance features from PG-CNN for guiding AG-GCN to exploit the contextual information ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , nearby objects) to enhance posture features. Then, we use the enhanced posture features from AG-GCN to help PG-CNN focus on critical local areas of video frames that are related to driver behaviors. In this sense, we are able to use the interaction between two modalities to extract more discriminative features and thus improve the recognition accuracy. Experimental results on Drive&Act dataset show that our method outperforms state-of-the-art methods by a large margin (67.83% vs. 63.64%). Furthermore, we collect a bus driver behavior recognition dataset and yield consistent performance gain against baseline methods, demonstrating the effectiveness of our method in real-world applications. The source code and trained models are available at github.com/SCUT-AILab/BPAI-Net/.

Driving Behavior Recognition Algorithm Combining Attention Mechanism and Lightweight Network.

In actual driving scenes, recognizing and preventing drivers’ non-standard driving behavior is helpful in reducing traffic accidents. To resolve the problems of various driving behaviors, a large range of action, and the low recognition accuracy of traditional detection methods, in this paper, a driving behavior recognition algorithm was proposed that combines an attention mechanism and lightweight network. The attention module was integrated into the YOLOV4 model after improving the feature extraction network, and the structure of the attention module was also improved. According to the 20,000 images of the Kaggle dataset, 10 typical driving behaviors were analyzed, processed, and recognized. The comparison and ablation experimental results showed that the fusion of an improved attention mechanism and lightweight network model had good performance in accuracy, model size, and FLOPs.