Driver Distraction Detection Research Articles

Detecting Driver Behavior Using Stacked Long Short Term Memory Network With Attention Layer

Driver distraction is one of the primary reasons for fatal car accidents. Modern cars with advanced infotainment systems often take some cognitive attention away from the road, consequently causing more distraction. Driver behavior analysis can be used to address the driver distraction problem. Three important features of intelligence and cognition are perception, attention and sensory memory. In this work, we use a stacked LSTM network with attention to detect driver distraction using driving data and compare this model with both stacked LSTM and MLP models to show the positive effect of using attention mechanism on the model's performance. We conducted an experiment with eight driving scenarios and collected a large dataset of driving data. First, an MLP was built to detect driver distraction. Next, we increased the intelligence level of the system by using an LSTM network. Third, we used the attention mechanism increment on the top of the LSTM model to enhance the model performance. We show that these three increments increase intelligence by reducing train and test error. The minimum train and test error of the stacked LSTM were 0.57 and 0.9 that were 0.4 less than the MLP minimum train and test error. Adding attention to the stacked LSTM model decreased the train and test error to 0.69 and 0.75. Results also show diminished the overfitting problem and reduction in computational expenses when adding attention.

Driver distraction detection using machine learning algorithms: an experimental approach

Driver distraction is the leading cause of accidents that contributes to 25% of all road crashes. In order to reduce the risks posed by distraction, the warning must be given to the driver once distraction is detected. According to the literature, no rankings of relevant features have been presented. In this study, the most relevant features in detecting driver distraction are identified in a closed testing environment. The relevant features are found to be the mean values of speed and lane deviation, maximum values of eye gaze in y direction, and head movement in x direction. After the relevant features have been identified, pre-processed data with relevant features are fed into decision tree classifiers to discriminate the data into normal and distracted driving. The results show that the detection accuracy of 78.4% using decision tree can be achieved. By eliminating unhelpful features, the time required to process data is reduced by around 40% to make the proposed technique suitable for real-time application.

Driver distraction detection using machine learning algorithms: an experimental approach

Driver distraction detection using machine learning algorithms: an experimental approach

Machine Learning and End-to-End Deep Learning for Monitoring Driver Distractions From Physiological and Visual Signals

It is only a matter of time until autonomous vehicles become ubiquitous; however, human driving supervision will remain a necessity for decades. To assess the driver's ability to take control over the vehicle in critical scenarios, driver distractions can be monitored using wearable sensors or sensors that are embedded in the vehicle, such as video cameras. The types of driving distractions that can be sensed with various sensors is an open research question that this study attempts to answer. This study compared data from physiological sensors (palm electrodermal activity (pEDA), heart rate and breathing rate) and visual sensors (eye tracking, pupil diameter, nasal EDA (nEDA), emotional activation and facial action units (AUs)) for the detection of four types of distractions. The dataset was collected in a previous driving simulation study. The statistical tests showed that the most informative feature/modality for detecting driver distraction depends on the type of distraction, with emotional activation and AUs being the most promising. The experimental comparison of seven classical machine learning (ML) and seven end-to-end deep learning (DL) methods, which were evaluated on a separate test set of 10 subjects, showed that when classifying windows into distracted or not distracted, the highest F1-score of 79% was realized by the extreme gradient boosting (XGB) classifier using 60-second windows of AUs as input. When classifying complete driving sessions, XGB's F1-score was 94%. The best-performing DL model was a spectro-temporal ResNet, which realized an F1-score of 75% when classifying segments and an F1-score of 87% when classifying complete driving sessions. Finally, this study identified and discussed problems, such as label jitter, scenario overfitting and unsatisfactory generalization performance, that may adversely affect related ML approaches.

Distracted Driver Detection with Deep Convolutional Neural Network

According to the World Health Organization (WHO), over 1.3 million deaths occur worldwide each year due to traffic accidents alone. This figure elevates traffic mishaps to be the eight leading cause of death. According to another study the United States National Highway Traffic Safety Administration (NHTSA), the major cause of road deaths and injury is distracted drivers. Motivated by recent advancement of deep learning and computer vision in predicting drivers’ behaviour, this paper attempts to investigate the optimal deep learning network architecture to accurately detect distracted drivers over visual feed. Specifically, a thorough evaluation and detailed benchmark comparisons of pretrained deep convolutional neural network is carried out. Results indicate that the proposed VGG16network architecture is capable of achieving 96% accuracy on the test dataset images.

Classification of Driver Distraction: A Comprehensive Analysis of Feature Generation, Machine Learning, and Input Measures.

The objective of this study was to analyze a set of driver performance and physiological data using advanced machine learning approaches, including feature generation, to determine the best-performing algorithms for detecting driver distraction and predicting the source of distraction. Distracted driving is a causal factor in many vehicle crashes, often resulting in injuries and deaths. As mobile devices and in-vehicle information systems become more prevalent, the ability to detect and mitigate driver distraction becomes more important. This study trained 21 algorithms to identify when drivers were distracted by secondary cognitive and texting tasks. The algorithms included physiological and driving behavioral input processed with a comprehensive feature generation package, Time Series Feature Extraction based on Scalable Hypothesis tests. Results showed that a Random Forest algorithm, trained using only driving behavior measures and excluding driver physiological data, was the highest-performing algorithm for accurately classifying driver distraction. The most important input measures identified were lane offset, speed, and steering, whereas the most important feature types were standard deviation, quantiles, and nonlinear transforms. This work suggests that distraction detection algorithms may be improved by considering ensemble machine learning algorithms that are trained with driving behavior measures and nonstandard features. In addition, the study presents several new indicators of distraction derived from speed and steering measures. Future development of distraction mitigation systems should focus on driver behavior-based algorithms that use complex feature generation techniques.

Detection and Evaluation of Driver Distraction Using Machine Learning and Fuzzy Logic

In addition to vehicle control, drivers often perform secondary tasks that impede driving. Reduction of driver distraction is an important challenge for the safety of intelligent transportation systems. In this paper, a methodology for the detection and evaluation of driver distraction while performing secondary tasks is described and an appropriate hardware and a software environment is offered and studied. The system includes a model of normal driving, a subsystem for measuring the errors from the secondary tasks, and a module for total distraction evaluation. A new machine learning algorithm defines driver performance in lane keeping and speed maintenance on a specific road segment. To recognize the errors, a method is proposed, which compares normal driving parameters with ones obtained while conducting a secondary task. To evaluate distraction, an effective fuzzy logic algorithm is used. To verify the proposed approach, a case study with driver-in-the-loop experiments was carried out, in which participants performed the secondary task, namely chatting on a cell phone. The results presented in this research confirm its capability to detect and to precisely measure a level of abnormal driver performance.

Real-time detection of driver distraction: random projections for pseudo-inversion-based neural training

There is an accumulating evidence that distracted driving is a leading cause of vehicle crashes and accidents. In order to support safe driving, numerous methods of detecting distraction have been proposed, which are empirically focused on certain driving contexts and gaze behaviour. This paper aims at illustrating a method for the non-intrusive and real-time detection of visual distraction based on vehicle dynamics data and environmental data, without using eye-tracker information. Experiments are carried out in the context of the automotive domain of the European project Holides, which addresses development and qualification of adaptive cooperative human–machine systems, and is co-funded by ARTEMIS Joint Undertaking and Italian University, Educational and Research Department. The collected data are analysed by a single-layer feedforward neural network trained through pseudo-inversion methods, characterized by direct determination of output weights given randomly set input weights and biases. One main feature of our work is the convenient setting of input weights by the so-called sparse random projections: the presence of a great number of null elements in the involved matrices makes especially parsimonious the use at run time of the trained network. Moreover, we use a genetic approach to better explore the input weights network space. The obtained results show better performance with respect to classical pseudo-inversion methods and effective and parsimonious use of memory resources.

Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad hoc methods are often used. In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically weighted ensemble of convolutional neural networks; we show that a weighted ensemble of classifiers using a genetic algorithm yields a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.

An Adaptive Forward Collision Warning Framework Design Based on Driver Distraction

Forward Collision Warning (FCW) is a promising Advanced Driver Assistance System (ADAS) to mitigate rear-end collisions. The deterministic FCW approaches may occasionally lead to the issuance of annoying false warnings, as they cannot be individualized for different drivers. This application oversight, which may cause the driver to deactivate the system, has been tackled with some adaptive methods. However, driver distraction, which is one of the most influential driver-specific factors on FCW warnings acceptability, has not been considered yet and is analyzed in this paper for the first time. Specifically, the adaptive FCW method proposed in this paper generates the warnings by continuously comparing Time Headway with a flexible threshold. The core of the proposed threshold updating mechanism is a real-time monitoring of the driver reactions against the previously generated warnings using the available indicators such as braking history. This method considers the driver distraction in parallel to fine-tune the calculated threshold in accordance with driver cognitive state. In order to incorporate the driver distraction in the system framework, a learning-based approach is designed which continuously estimates driver distraction by the virtue of different available Controller Area Network (CAN) bus time series, such as throttle pedal position, velocity, acceleration, and yaw rate. Neural network, as a widely adopted classification method, is nominated to detect driver distraction. The framework performance is evaluated over two realistic driving datasets. An approximately 80% false warning reduction is observed in analyzed safe scenarios, while no critical warning is missed in the dangerous ones.

Feature Based Techniques for a Driver's Distraction Detection using Supervised Learning Algorithms based on Fixed Monocular Video Camera

Most of the accidents occur due to drowsiness while driving, avoiding road signs and due to driver's distraction. Driver's distraction depends on various factors which include talking with passengers while driving, mood disorder, nervousness, anger, over-excitement, anxiety, loud music, illness, fatigue and different driver’s head rotations due to change in yaw, pitch and roll angle. The contribution of this paper is two-fold. Firstly, a data set is generated for conducting different experiments on driver’s distraction. Secondly, novel approaches are presented that use features based on facial points; especially the features computed using motion vectors and interpolation to detect a special type of driver’s distraction, i.e., driver’s head rotation due to change in yaw angle. These facial points are detected by Active Shape Model (ASM) and Boosted Regression with Markov Networks (BoRMaN). Various types of classifiers are trained and tested on different frames to decide about a driver's distraction. These approaches are also scale invariant. The results show that the approach that uses the novel ideas of motion vectors and interpolation outperforms other approaches in detection of driver’s head rotation. We are able to achieve a percentage accuracy of 98.45 using Neural Network.

Real‐time detection of distracted driving based on deep learning

Driver distraction is a leading factor in car crashes. With a goal to reduce traffic accidents and improve transportation safety, this study proposes a driver distraction detection system which identifies various types of distractions through a camera observing the driver. An assisted driving testbed is developed for the purpose of creating realistic driving experiences and validating the distraction detection algorithms. The authors collected a dataset which consists of images of the drivers in both normal and distracted driving postures. Four deep convolutional neural networks including VGG-16, AlexNet, GoogleNet, and residual network are implemented and evaluated on an embedded graphic processing unit platform. In addition, they developed a conversational warning system that alerts the driver in real-time when he/she does not focus on the driving task. Experimental results show that the proposed approach outperforms the baseline one which has only 256 neurons in the fully-connected layers. Furthermore, the results indicate that the GoogleNet is the best model out of the four for distraction detection in the driving simulator testbed.

Visual-Manual Distraction Detection Using Driving Performance Indicators With Naturalistic Driving Data

This paper investigates the problem of driver distraction detection using driving performance indicators from onboard kinematic measurements. First, naturalistic driving data from the integrated vehicle-based safety system program are processed, and cabin camera data are manually inspected to determine the driver’s state (i.e., distracted or attentive). Second, existing driving performance metrics, such as steering entropy, steering wheel reversal rate, and lane offset variance, are reviewed against the processed naturalistic driving data. Furthermore, a nonlinear autoregressive exogenous (NARX) driving model is developed to predict vehicle speed based on the range (distance headway), range rate, and speed history. For each driver, the NARX model is then trained on the attentive driving data. We show that the prediction error is correlated with driver distraction. Finally, two features, steering entropy and mean absolute speed prediction error from the NARX model are selected, and a support vector machine is trained to detect driving distraction. Prediction performances are reported.

Real-time detection of drivers’ texting and eating behavior based on vehicle dynamics

With rapid advancement in cellphones and intelligent in-vehicle technologies along with driver’s inclination to multitasking, crashes due to distracted driving had become a growing safety concern in our road network. Some previous studies attempted to detect distracted driving behaviors in real-time to mitigate their adverse consequences. However, these studies mainly focused on detecting either visual or cognitive distractions only, while most of the real-life distracting tasks involve driver’s visual, cognitive, and physical workload, simultaneously. Additionally, previous studies frequently used eye, head, or face tracking data, although current vehicles are not commonly equipped with technologies to acquire such data. Also those data are comparatively difficult to acquire in real-time during traffic monitoring operations. To address the above issues, this study focused on developing algorithms for detecting distraction tasks that involve simultaneous visual, cognitive, and physical workload using only vehicle dynamics data. Specifically, algorithms were developed to detect driving behaviors under two distraction tasks – texting and eating. Experiment was designed to include the two distracted driving scenarios and a control with multiple runs for each. A medium fidelity driving simulator was used for acquiring vehicle dynamics data for each scenario and each run. Several data mining techniques were explored in this study to investigate their performance in detecting distraction. Among them, the performance of two linear (linear discriminant analysis and logistic regression) and two nonlinear models (support vector machines and random forests) is reported in this article. Random forests algorithms had the best performance, which detected texting and eating distraction with an accuracy of 85.38% and 81.26%, respectively. This study may provide useful guidance to successful development and implementation of distracted driver detection algorithms in connected vehicle environment, as well as to auto manufacturers interested in integrating distraction detection systems in their vehicles.

ECG-based driver inattention identification during naturalistic driving using Mel-frequency cepstrum 2-D transform and convolutional neural networks

Abstract Driver distraction is a major cause of road accidents which can lead to severe physical injuries and death. Statistics indicate the need of a reliable driver distraction system which can monitor the driver׳s distraction in real time and alert the driver before a mishap happens. Early detection of driver distraction can help decrease the costs of roadway disasters. There has been large research efforts towards analyzing driving behavior or monitoring driver distraction using camera-based techniques. However, camera-based systems pose challenges such as privacy and latency in the signs of distraction. Physiological signals such as Electrocardiogram (ECG) and heart rate related measures have been extensively used to monitor human state at a physiological level and developed systems which alert divers well in advance. In this paper, we introduce an ECG based driver distraction detection system using Mel-frequency cepstrum representation and convolutional neural networks (CNN). The proposed model operates by feeding a two dimensional Mel-frequency cepstrum representation of ECG data as input to deep convolutional neural networks. We present a recipe to extract Mel frequency filter bank coefficients in time and frequency domains. The deep CNN is structured to automatically learn reliable discriminative patterns in the 2D spectro-temporal space as features thus replacing the traditional ad hoc hand-crafted features when working with the recorded time-series dataset. The classification accuracy of the proposed prediction algorithm was evaluated for ECG signals recorded from 10 subjects. The subjects aged 24 to 45, actively participated in the naturalistic driving experiment during the ECG recordings. The experimental results demonstrate that the proposed algorithm achieves a significant classification accuracy for across the subjects.

Towards online detection of driver distraction: Eye-movement simulation based on a combination of vestibulo–ocular reflex and optokinetic reflex models

With the aim of developing a method for evaluating driver distraction based on the difference between simulated and observed eye movements, we integrated a vestibulo–ocular reflex (VOR) model and an optokinetic reflex (OKR) model. On the basis of 2 × 2 factorial design analysis (VOR model only and VOR+OKR model; with and without visual stimulus), we confirmed that the VOR+OKR model showed a better performance than the VOR model, with a smaller mean-square error. We also conducted an experiment that used an n-back task to simulate mental workload while driving. The mean-square error between the simulated and observed eye movement became larger in the presence of mental workload. This result suggests that the VOR+OKR model could be applied in the evaluation of driver distraction.

Wearable Driver Distraction Identification On-The-Road via Continuous Decomposition of Galvanic Skin Responses.

One of the main reasons for fatal accidents on the road is distracted driving. The continuous attention of an individual driver is a necessity for the task of driving. While driving, certain levels of distraction can cause drivers to lose their attention, which might lead to an accident. Thus, the number of accidents can be reduced by early detection of distraction. Many studies have been conducted to automatically detect driver distraction. Although camera-based techniques have been successfully employed to characterize driver distraction, the risk of privacy violation is high. On the other hand, physiological signals have shown to be a privacy preserving and reliable indicator of driver state, while the acquisition technology might be intrusive to drivers in practical implementation. In this study, we investigate a continuous measure of phasic Galvanic Skin Responses (GSR) using a wristband wearable to identify distraction of drivers during a driving experiment on-the-road. We first decompose the raw GSR signal into its phasic and tonic components using Continuous Decomposition Analysis (CDA), and then the continuous phasic component containing relevant characteristics of the skin conductance signals is investigated for further analysis. We generated a high resolution spectro-temporal transformation of the GSR signals for non-distracted and distracted (calling and texting) scenarios to visualize the associated behavior of the decomposed phasic GSR signal in correlation with distracted scenarios. According to the spectrogram observations, we extract relevant spectral and temporal features to capture the patterns associated with the distracted scenarios at the physiological level. We then performed feature selection using support vector machine recursive feature elimination (SVM-RFE) in order to: (1) generate a rank of the distinguishing features among the subject population, and (2) create a reduced feature subset toward more efficient distraction identification on the edge at the generalization phase. We employed support vector machine (SVM) to generate the 10-fold cross validation (10-CV) identification performance measures. Our experimental results demonstrated cross-validation accuracy of 94.81% using all the features and the accuracy of 93.01% using reduced feature space. The SVM-RFE selected set of features generated a marginal decrease in accuracy while reducing the redundancy in the input feature space toward shorter response time necessary for early notification of distracted state of the driver.

A Novel Driver Performance Model Based on Machine Learning

Models of road vehicle driver behaviour are widely used in several disciplines, like driver distraction and autonomous driving. In this paper, a novel driver performance model, which is unique for every driver, is introduced. The driver is modelled with machine learning algorithms, namely artificial neural network and adaptive neuro-fuzzy inference system. Every model is trained and validated with the data collected during the real-time driver-in-the-loop experiment on a vehicle simulator for each driver separately. In total, 18 participants contributed to the experiment. Although the prediction accuracy of the models depends on the algorithm specifications, the artificial neural network was slightly more accurate in driver performance prediction comparing to the adaptive neuro-fuzzy inference system. The driver models may be used in detection of driver distraction induced by in-vehicle information system.

Estimation of Lead Vehicle Kinematics Using Camera-Based Data for Driver Distraction Detection

Estimation of Lead Vehicle Kinematics Using Camera-Based Data for Driver Distraction Detection

Examining drivers' eye glance patterns during distracted driving: Insights from scanning randomness and glance transition matrix

IntroductionVisual attention to the driving environment is of great importance for road safety. Eye glance behavior has been used as an indicator of distracted driving. This study examined and quantified drivers' glance patterns and features during distracted driving. MethodData from an existing naturalistic driving study were used. Entropy rate was calculated and used to assess the randomness associated with drivers' scanning patterns. A glance-transition proportion matrix was defined to quantity visual search patterns transitioning among four main eye glance locations while driving (i.e., forward on-road, phone, mirrors and others). All measurements were calculated within a 5s time window under both cell phone and non-cell phone use conditions. ResultsResults of the glance data analyses showed different patterns between distracted and non-distracted driving, featured by a higher entropy rate value and highly biased attention transferring between forward and phone locations during distracted driving. Drivers in general had higher number of glance transitions, and their on-road glance duration was significantly shorter during distracted driving when compared to non-distracted driving. ConclusionsResults suggest that drivers have a higher scanning randomness/disorder level and shift their main attention from surrounding areas towards phone area when engaging in visual-manual tasks. Practical applicationsDrivers' visual search patterns during visual-manual distraction with a high scanning randomness and a high proportion of eye glance transitions towards the location of the phone provide insight into driver distraction detection. This will help to inform the design of in-vehicle human-machine interface/systems.