Driving Behavior Recognition Research Articles

3D skeleton aware driver behavior recognition framework for autonomous driving system

The recognition of the driver’s behaviors inside an autonomous vehicle can effectively address emergency handling in autonomous driving and is crucial for ensuring the driver’s safety. Driver behavior recognition is a challenging task due to factors such as variations, diversities, complexities, and strong interferences in behaviors. In this paper, to realize the application in the autonomous driving scenes, a novel 3D skeleton aware behavior recognition framework is proposed to recognize various driver behaviors in autonomous driving systems. First, a 3D human pose estimation network (Pose-GTFNet) with temporal Transformer and spatial graph convolutional network (GCN) is designed to infer 3D human poses from 2D pose sequences. Second, based on the obtained 3D human pose sequences, a behavior recognition network (Beh-MSFNet) with multi-skeleton feature fusion is designed to recognize driver behaviors. In the experiments, the Pose-GTFNet and Beh-MSFNet can get the best performance compared with most state-of-the-art (SOTA) methods on the Human3.6M human pose dataset, JHMDB and SHREC action recognition dataset, respectively. In addition, the proposed driver behavior recognition framework can achieve SOTA performance on the Drive&Act and Driver-Skeleton driver behavior datasets.

Driver behavior recognition based on dual-branch and deformable convolutional network method

Aiming at the task of driver behavior recognition in the car cockpit, this paper proposes a recognition method based on a dual-branch neural network. The main branch of the network model uses ResNet50 as the backbone network for feature extraction, and uses deformable convolution to adapt the model to the shape and position changes of the driver in the image. The auxiliary branch assists in updating the parameters of the backbone network during the gradient backpropagation process, so that the backbone network can better extract features that are conducive to driver behavior recognition, thereby improving the recognition performance of the model. The ablation experiment and comparative experiment results of the network model on the State Farm public dataset show that the recognition accuracy of the proposed network model can reach 96.23%, and the recognition effect is better for easily confused behavior categories. The research results are of great significance for understanding driver behavior in the car cockpit and ensuring driving safety.

Vehicle Driving Behavior Recognition and Optimization Strategies Based on Cloud Computing and SSA-BP Algorithm

Vehicle Driving Behavior Recognition and Optimization Strategies Based on Cloud Computing and SSA-BP Algorithm

Taxonomy, Open Challenges, Motivations, and Recommendations in Driver Behavior Recognition: A Systematic Review

Taxonomy, Open Challenges, Motivations, and Recommendations in Driver Behavior Recognition: A Systematic Review

Appearance-posture fusion network for distracted driving behavior recognition

In recent years, detection techniques using computer vision and deep learning have shown promise in assessing driver distraction. This paper proposes a fusion network that combines a Spatial-Temporal Graph Convolutional Network (ST-GCN) and a hybrid convolutional network to integrate multimodal input data for recognizing distracted driver behavior. Specifically, to address the limitations of the ST-GCN method in modeling long-distance joint interaction features and inadequate temporal feature extraction, we design the Spatially Symmetric Configuration Partitioning Graph Convolutional Network (SSCP-GCN) to model relative motion information of symmetric relationships between limbs. Specifically, we utilize densely connected blocks for processing multi-scale temporal information between consecutive frames, thereby enhancing the reuse of bottom features. Furthermore, the expression of important temporal information is augmented by the introduction of the channel attention mechanism. To tackle the problem that the Mixed Convolution (MC) combining 3D convolution with 2D convolution cannot extract higher-order timing information and has limitations in modeling global dependency relationships, we compensate for its inability to capture higher-order temporal semantic information using the Time Shift Module (TSM) without consuming additional computational resources. Additionally, the 3D Multi-Head Self-Attention mechanism (3D MHSA) is employed to integrate global spatial–temporal information of high-level features, avoiding the issue of model complexity proliferation caused by the deep stacking design of Convolutional Neural Networks (CNN). Lastly, we introduce a multistream network framework that integrates driver posture and appearance features to harness complementary advantages, enabling us to combine multimodal input features to achieve better model performance. Experimental results indicate that the accuracy of the network designed in this paper reaches 95.6% and 94.3% on ASU dataset and NTU-RGB+D dataset, respectively. The small size of the model offers the possibility for practical application of the algorithm.

Recognition of aggressive driving behavior under abnormal weather based on Convolutional Neural Network and transfer learning

Objectives Aggressive driving behavior can lead to potential traffic collision risks, and abnormal weather conditions can exacerbate this behavior. This study aims to develop recognition models for aggressive driving under various climate conditions, addressing the challenge of collecting sufficient data in abnormal weather. Methods Driving data was collected in a virtual environment using a driving simulator under both normal and abnormal weather conditions. A model was trained on data from normal weather (source domain) and then transferred to foggy and rainy weather conditions (target domains) for retraining and fine-tuning. The K-means algorithm clustered driving behavior instances into three styles: aggressive, normal, and cautious. These clusters were used as labels for each instance in training a CNN model. The pre-trained CNN model was then transferred and fine-tuned for abnormal weather conditions. Results The transferred models showed improved recognition performance, achieving an accuracy score of 0.81 in both foggy and rainy weather conditions. This surpassed the non-transferred models’ accuracy scores of 0.72 and 0.69, respectively. Conclusions The study demonstrates the significant application value of transfer learning in recognizing aggressive driving behaviors with limited data. It also highlights the feasibility of using this approach to address the challenges of driving behavior recognition under abnormal weather conditions.

Improving night driving behavior recognition with ResNet50

The issue of driving behavior at night poses significant challenges due to reduced visibility and increased risk of accidents. Recent works have leveraged deep learning techniques to enhance night-time driving safety. However, the limited availability of high-quality training data and the lack of robustness in existing models present significant problems. In this work, we propose a novel approach to improve driving behavior recognition at night using ResNet50 with contrast limited adapted histogram equalization (CLAHE). We collected a new dataset and developed a more effective and robust model that can accurately recognize driving behaviors under low-illumination conditions, thereby reducing the likelihood of collisions and improving overall road safety. The experimental results demonstrate significant improvements in the deep learning model’s performance compared to conventional methods. Notably, the ResNet50 model delivers the best performance with accuracy rates of 90.73% using NIGHT-VIS-CLAHE data, demonstrating a 16% improvement in accuracy. For benchmark purposes, the InceptionV3, GoogleNet, and MobileNetV2 models also show enhanced accuracy through CLAHE implementation. Furthermore, NIGHT-VIS-CLAHE implementation in ResNet50 achieved 90.29% accuracy, surpassing the best NIGHT-IR InceptionV3 at 89.27%, highlighting the advantage of ResNet50 with CLAHE in low-light conditions even against infra-red sensor.

A systematic review for the fatigue driving behavior recognition method

Fatigue driving is one of the primary causative factors of road accidents. It is of great significance to discern, identify and warn drivers in time for traffic safety and reduce traffic accidents. In this paper, a systematic review for the fatigue driving behavior recognition method is developed to analyze its research status and development trends. Firstly, the data information and its application scenarios related to fatigue driving is detailed. Three driving behavior recognition methods based on different types of signal data are summarized and analyzed, and this signal data can be divided into physiological signal characteristics, visual signal characteristics, vehicle sensor data characteristics and multi-data information fusion. By summarizing and comparing the recognition effect of existing fatigue driving recognition methods, combined with deep learning technology, the paper concludes the fatigue driving behavior recognition method based on single data source has some shortcomings such as low accuracy and easy to be affected by external factors, but the recognition method based on multi-feature information fusion can achieve a exhilarated recognition result. Finally, some prospects are given to analyze the development trend of fatigue driving behavior recognition in the future.

FDAN: Fuzzy deep attention networks for driver behavior recognition

Driver behavior is an essential factor affecting traffic safety, and driver behavior monitoring systems (DMSs) are widely exploited in intelligent transportation systems to reduce the risk of traffic accidents. However, understanding driver behavior is challenging because of the uncertainty of real driving scenarios. Most of the existing methods use deterministic models, which suffer from data uncertainty, for recognizing driver behaviors. In this paper, the fuzzy deep attention network (FDAN) method is proposed to improve driver behavior recognition. FDAN integrates fuzzy logic and an attention mechanism into deep neural networks, which enhances the representation ability of the model and reduces the uncertainty of the data. The attention mechanism with a lightweight squeeze-and-excitation block is embedded in the deep learning model for adaptively refining features. A DMS is designed, and the distracted driver dataset from the real scene is built. Experimental results confirm the proposed method performs better than the existing methods.

Deep Unsupervised Transfer Adversarial Network for Abnormal Driving Behavior Recognition Based on Smartphone Sensors

Deep Unsupervised Transfer Adversarial Network for Abnormal Driving Behavior Recognition Based on Smartphone Sensors

Human-Machine Shared Control for Industrial Vehicles: A Personalized Driver Behavior Recognition and Authority Allocation Scheme

Human-Machine Shared Control for Industrial Vehicles: A Personalized Driver Behavior Recognition and Authority Allocation Scheme

An Effective Multi-Scale Framework for Driver Behavior Recognition With Incomplete Skeletons

An Effective Multi-Scale Framework for Driver Behavior Recognition With Incomplete Skeletons

Distracted driver behavior recognition using modified capsule networks

Human activity recognition (HAR) is an increasingly active study field within the computer vision community. In HAR, driver behavior can be detected to ensure safe travel. Detect driver behaviors using a capsule network with leave-one-subject-out validation. The study was done using CapsNet with leave-one-subject-out validation to identify driving habits. The proposed method in this study consists of two parts, namely encoder and decoder. The encoder used in this study modifies Sabour’s capsule network architecture by adding a convolution layer before going to the primary capsule layer. The proposed method is evaluated using a primary dataset with 10 classes and 300 images for each class. The dataset is split based on hold-out validation and leave-one-subject-out validation. The resulting models were then compared to conventional CNN architecture. The objective of the research is to identify driving behavior. In this study, the proposed method results an accuracy rate of 97.83 % in the split dataset using hold-out validation. However, the accuracy decreased by 53.11 % when the proposed method was used on a split dataset using leave-one-subject-out validation. This is because the proposed method extracts all features including the attributes of each participant contained in the input image (user-independent). Thus, the resulting model in this study tends to overfit.

Driver distraction detection using semi-supervised lightweight vision transformer

The continuously increasing number of traffic accidents necessitates addressing distracted driving, which is responsible for numerous fatalities. Enhancing driver behavior recognition, particularly through developing a highly reliable Advanced Driver Assistance System (ADAS), holds substantial potential for cultivating safer transportation systems. Inspired by the success of Convolutional Neural Networks (CNNs), various networks have been proposed to enhance the detection accuracy of distracting behaviors. However, existing models have too many parameters and rely heavily on extensive labeled data, leading to time-consuming labeling and large models. To address these limitations, we propose a distracted behavior detection method based on a lightweight vision transformer trained using pseudo-label-based semi-supervised learning to learn discriminative representations from labeled and unlabeled data while ensuring a small model and speedy inference. Specifically, we create strong and weak augmented versions of the input minibatch and employ a hybrid lightweight transformer model in a teacher-student network. Pseudo-labels are generated from the teacher's predictions on weakly augmented data. The student model aligns with these labels on strongly augmented input, with teacher parameters evolving through exponential moving average. Our method presents a real-time, accurate solution for distracted driver detection, with the potential to significantly enhance road safety by reducing accidents. The effectiveness of the proposed approach is demonstrated through a comparative evaluation against alternative fully-supervised and semi-supervised methods. Furthermore, our method is evaluated in naturalistic driving settings with varying lighting and complex backgrounds. Experiments conducted on two publicly available driver distraction detection datasets show that our method outperforms current state-of-the-art approaches.

Deformation Gated Recurrent Network for Lane-Level Abnormal Driving Behavior Recognition

As a significant part of traffic accident prevention, abnormal driving behavior recognition has been receiving extensive attention. However, the granularity of existing abnormal driving behavior recognition is mostly at road-level, and these methods’ high complexity leads to high overhead on training and recognition. In this article, we propose a deformation gated recurrent network for lane-level abnormal driving behavior recognition. Firstly, we use conditional random field model to calculate the lane change necessity of the vehicle, which helps us to distinguish whether the lane-changing behavior is reasonable. Secondly, we propose deformation gated recurrent network (DF-GRN) and trajectory entropy to capture the implicit relationship between trajectories and shorten recognition time. Finally, we get classified results including aggressive, distracted and normal driving behavior from the network. Distracted and aggressive behavior will be marked as anomaly. The effectiveness and real-time nature of the network are verified by experiments on Hangzhou and Chengdu location datasets.

Research on Lightweight-Based Algorithm for Detecting Distracted Driving Behaviour

In order to solve the existing distracted driving behaviour detection algorithms’ problems such as low recognition accuracy, high leakage rate, high false recognition rate, poor real-time performance, etc., and to achieve high-precision real-time detection of common distracted driving behaviours (mobile phone use, smoking, drinking), this paper proposes a driver distracted driving behaviour recognition algorithm based on YOLOv5. Firstly, to address the problem of poor real-time identification, the computational and parametric quantities of the network are reduced by introducing a lightweight network, Ghostnet. Secondly, the use of GSConv reduces the complexity of the algorithm and ensures that there is a balance between the recognition speed and accuracy of the algorithm. Then, for the problem of missed and misidentified cigarettes during the detection process, the Soft-NMS algorithm is used to reduce the problems of missed and false detection of cigarettes without changing the computational complexity. Finally, in order to better detect the target of interest, the CBAM is utilised to enhance the algorithm’s attention to the target of interest. The experiments show that on the homemade distracted driving behaviour dataset, the improved YOLOv5 model improves the mAP@0.5 of the YOLOv5s by 1.5 percentage points, while the computational volume is reduced by 7.6 GFLOPs, which improves the accuracy of distracted driving behaviour recognition and ensures the real-time performance of the detection speed.

Identification of aggressive driving behavior of online car‐hailing drivers based on association classification

AbstractWith the rapid development of online car‐hailing, the related crashes have become a key issue with public concern. Identifying and predicting aggressive driving behaviors is critical to reduce traffic crashes. In this study, we propose a method to recognize aggressive driving behavior based on association classification, with multisource features being employed, including driver emotion, vehicle kinematic characteristics, and road environment. The model performs best in a 10‐fold cross‐test when the minimum support and minimum confidence are set as 0.01 and 0.8, respectively. Besides, we also compare the performance of aggressive driving behavior recognition classifiers constructed using association classification with other rule‐based classification methods, including ID3, C4.5, CART, and Random Forest. The results show that association classification performs better than other classification competitors. Thirty‐six if–then rules generated by the association classification are used to analyze the influencing factors and associated mechanisms of aggressive driving behavior. It is found that aggressive driving behavior is highly correlated with driver anger and disgust emotions. Aggressive driving behavior is more likely to occur when no passengers are in the car than the case with passengers. Driver entertainment behavior and passenger interference also affect driving behavior. Moreover, drivers are prone to aggressive driving when making a U‐turn. This research not only proposed a new identification method for aggressive driving behavior but also provided a comprehensive understanding of the associated influencing factors which thus benefit the further research and development of safety assistance driving devices.

Distracted driving behavior recognition based on improved MobileNetV2

Distracted driving behavior recognition based on improved MobileNetV2

Dangerous Driving Behavior Recognition Based on YOLO Algorithm and BP Neural Network

Dangerous Driving Behavior Recognition Based on YOLO Algorithm and BP Neural Network

Fusion of Gaze and Scene Information for Driving Behaviour Recognition: A Graph-Neural-Network- Based Framework

Accurate recognition of driver behaviours is the basis for a reliable driver assistance system. This paper proposes a novel fusion framework for driver behaviour recognition that utilises the traffic scene and driver gaze information. The proposed framework is based on the graph neural network (GNN) and contains three modules, namely, the gaze analysing (GA) module, scene understanding (SU) module and the information fusion (IF) module. The GA module is used to obtain gaze images of drivers, and extract the gaze features from the images. The SU module provides trajectory predictions for surrounding vehicles, motorcycles, bicycles and other traffic participants. The GA and SU modules are parallel and the outputs of both modules are sent to the IF module that fuses the gaze and scene information using the attention mechanism and recognises the driving behaviours through a combined classifier. The proposed framework is verified on a naturalistic driving dataset. The comparative experiments with the state-of-the-art methods demonstrate that the proposed framework has superior performance for driving behaviour recognition in various situations.