Human-centric Driver Assistance Systems Research Articles

Decision-making in driver-automation shared control: A review and perspectives

Shared control schemes allow a human driver to work with an automated driving agent in driver-vehicle systems while retaining the driver's abilities to control. The human driver, as an essential agent in the driver-vehicle shared control systems, should be precisely modeled regarding their cognitive processes, control strategies, and decision-making processes. The interactive strategy design between drivers and automated driving agents brings an excellent challenge for human-centric driver assistance systems due to the inherent characteristics of humans. Many open-ended questions arise, such as what proper role of human drivers should act in a shared control scheme? How to make an intelligent decision capable of balancing the benefits of agents in shared control systems? Due to the advent of these attentions and questions, it is desirable to present a survey on the decision-making between human drivers and highly automated vehicles, to understand their architectures, human driver modeling, and interaction strategies under the driver-vehicle shared schemes. Finally, we give a further discussion on the key future challenges and opportunities. They are likely to shape new potential research directions.

A Probabilistic Approach to Measuring Driving Behavior Similarity With Driving Primitives

Evaluating the similarity levels of driving behavior plays a pivotal role in driving style classification and analysis, thus benefiting the design of human-centric driver assistance systems. This article presents a novel framework capable of quantitatively measuring the similarity of driving behaviors for human based on driving primitives, i.e., the building blocks of driving behavior. To this end, we develop a Bayesian nonparametric method by integrating hierarchical Dirichlet process (HDP) with a hidden Markov model (HMM) in order to automatically extract the driving primitives from sequential observations without using any prior knowledge. Then, we propose a grid-based relative entropy approach, which allows quantifying the probabilistic similarity levels among these extracted primitives. Finally, the naturalistic driving data from 10 drivers are collected to evaluate the proposed framework, with comparison to traditional work. Experimental results demonstrate that the proposed probabilistic framework based on driving primitives can provide a quantitative measurement of similar levels of driving behavior associated with the dynamic and stochastic characteristics.

Detecting Driver Normal and Emergency Lane-Changing Intentions With Queuing Network-Based Driver Models

Driver intention detection is an important component in human-centric driver assistance systems. This article proposes a novel method for detecting driver normal and emergency left- or right-lane-changing intentions by using driver models based on the queuing network cognitive architecture. Driver lane-changing and lane-keeping models are developed and used to simulate driver behavior data associated with 5 kinds of intentions (i.e., normal and emergency left- or right-lane-changing and lane-keeping intentions). The differences between 5 sets of simulated behavior data and the collected actual behavior data are computed, and the intention associated with the smallest difference is determined as the detection outcome. The experimental results from 14 drivers in a driving simulator show that the method can detect normal and emergency lane-changing intentions within 0.325 s and 0.268 s of the steering maneuver onset, respectively, with high accuracy (98.27% for normal lane changes and 90.98% for emergency lane changes) and low false alarm rate (0.294%).

Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian Approach

Reports of traffic accidents show that a considerable percentage of the accidents are caused by human factors. Human-centric driver assistance systems, with integrated sensing, processing and networking, aim to find solutions to this problem and other relevant issues. The key technology in such systems is the capability to automatically understand and characterize driver behaviors. In this paper, we propose a novel, efficient feature extraction approach for driving postures from a video camera, which consists of Homomorphic filter, skin-like regions segmentation, canny edge detection, connected regions detection, small connected regions deletion and spatial scale ratio calculation. With features extracted from a driving posture dataset we created at Southeast University (SEU), holdout and cross-validation experiments on driving posture classification are then conducted using Bayes classifier. Compared with a number of commonly used classification methods including naive Bayes classifier, subspace classifier, linear perception classifier and Parzen classifier, the holdout and cross-validation experiments show that the Bayes classifier offers better classification performance than the other four classifiers. Among the four predefined classes, i.e., grasping the steering wheel, operating the shift gear, eating a cake and talking on a cellular phone, the class of talking on a cellular phone is the most difficult to classify. With Bayes classifier, the classification accuracies of talking on a cellular phone are over 90 % in holdout and cross-validation experiments, which shows the effectiveness of the proposed feature extraction method and the importance of Bayes classifier in automatically understanding and characterizing driver behaviors towards human-centric driver assistance systems.

Recognition of driving postures by multiwavelet transform and multilayer perceptron classifier

To develop Human-centric Driver Assistance Systems (HDAS) for automatic understanding and charactering of driver behaviors, an efficient feature extraction of driving postures based on Geronimo–Hardin–Massopust (GHM) multiwavelet transform is proposed, and Multilayer Perceptron (MLP) classifiers with three layers are then exploited in order to recognize four pre-defined classes of driving postures. With features extracted from a driving posture dataset created at Southeast University (SEU), the holdout and cross-validation experiments on driving posture classification are conducted by MLP classifiers, compared with the Intersection Kernel Support Vector Machines (IKSVMs), the k-Nearest Neighbor (kNN) classifier and the Parzen classifier. The experimental results show that feature extraction based on GHM multwavelet transform and MLP classifier, using softmax activation function in the output layer and hyperbolic tangent activation function in the hidden layer, offer the best classification performance compared to IKSVMs, kNN and Parzen classifiers. The experimental results also show that talking on a cellular phone is the most difficult one to classify among four predefined classes, which are 83.01% and 84.04% in the holdout and cross-validation experiments respectively. These results show the effectiveness of the feature extraction approach using GHM multiwavelet transform and MLP classifier in automatically understanding and characterizing driver behaviors towards Human-centric Driver Assistance Systems (HDAS).

Erratum to: Recognition of driving postures by combined features and random subspace ensemble of multilayer perceptron classifiers

Human-centric driver assistance systems with integrated sensing, processing and networking aim to find solutions for traffic accidents and other relevant issues. The key technology for developing such a system is the capability of automatically understanding and characterizing driver behaviors. This paper proposes a novel driving posture recognition approach, which consists of an efficient combined feature extraction and a random subspace ensemble of multilayer perceptron classifiers. A Southeast University Driving Posture Database (SEU-DP Database) has been created for training and testing the proposed approach. The data set contains driver images of (1) grasping the steering wheel, (2) operating the shift lever, (3) eating a cake and (4) talking on a cellular phone. Combining spatial scale features and histogram-based features, holdout and cross-validation experiments on driving posture classification are conducted, comparatively. The experimental results indicate that the proposed combined feature extraction approach with random subspace ensemble of multilayer perceptron classifiers outperforms the two individual feature extraction approaches. The experiments also suggest that talking on a cellular phone is the most difficult posture in classification among the four predefined postures. Using the proposed approach, the classification accuracy on talking on a cellular phone is over 89 % in both holdout and cross-validation experiments. These results show the effectiveness of the proposed combined feature extraction approach and random subspace ensemble of multilayer perceptron classifiers in automatically understanding and characterizing driver behaviors toward human-centric driver assistance systems.

Recognition of driving postures by contourlet transform and random forests

An efficient feature extraction approach for driving postures from a video camera, which consists of Homomorphic filtering, skin-like regions segmentation and contourlet transform (CT), was proposed. With features extracted from a driving posture dataset created at Southeast University (SEU), holdout and cross-validation experiments on driving posture classification were then conducted using random forests (RF) classifier. Compared with a number of commonly used classification methods including linear perceptron classifier, k-nearest-neighbour classifier and multilayer perceptron (MLP) classifier, the experiments results showed that the RF classifier offers the best classification performance among the four classifiers. Among the four predefined classes, that is, grasping the steering wheel, operating the shift gear, eating and talking on a cellular phone, the class of eating is the most difficult to classify. With RF classifier, the classification accuracies of eating are over 88% in holdout and cross-validation experiments, thus demonstrating the effectiveness of the proposed feature extraction method and the importance of RF classifier in automatically understanding and characterising driver's behaviours towards human-centric driver assistance systems.