Distraction Detection Research Articles

Detecting Human Driver Inattentive and Aggressive Driving Behavior Using Deep Learning: Recent Advances, Requirements and Open Challenges

Human drivers have different driving styles, experiences, and emotions due to unique driving characteristics, exhibiting their own driving behaviors and habits. Various research efforts have approached the problem of detecting abnormal human driver behavior with the aid of capturing and analyzing the face of driver and vehicle dynamics via image and video processing but the traditional methods are not capable of capturing complex temporal features of driving behaviors. However, with the advent of deep learning algorithms, a significant amount of research has also been conducted to predict and analyze driver's behavior or action related information using neural network algorithms. In this paper, we contribute to first classify and discuss Human Driver Inattentive Driving Behavior (HIDB) into two major categories, Driver Distraction (DD), Driver Fatigue (DF), or Drowsiness (DFD). Then we discuss the causes and effects of another human risky driving behavior called Aggressive Driving behavior (ADB). Aggressive driving Behavior (ADB) is a broad group of dangerous and aggressive driving styles that lead to severe accidents. Human abnormal driving behaviors DD, DFD, and ADB are affected by various factors including driver experience/inexperience of driving, age, and gender or illness. The study of the effects of these factors that may lead to deterioration in the driving skills and performance of a human driver is out of the scope of this paper. After describing the background of deep learning and its algorithms, we present an in-depth investigation of most recent deep learning-based systems, algorithms, and techniques for the detection of Distraction, Fatigue/Drowsiness, and Aggressiveness of a human driver. We attempt to achieve a comprehensive understanding of HIADB detection by presenting a detailed comparative analysis of all the recent techniques. Moreover, we highlight the fundamental requirements. Finally, we present and discuss some significant and essential open research challenges as future directions.

Driver distraction detection using machine learning algorithms: an experimental approach

Driver distraction is the leading cause of accidents that contributes to 25% of all road crashes. In order to reduce the risks posed by distraction, the warning must be given to the driver once distraction is detected. According to the literature, no rankings of relevant features have been presented. In this study, the most relevant features in detecting driver distraction are identified in a closed testing environment. The relevant features are found to be the mean values of speed and lane deviation, maximum values of eye gaze in y direction, and head movement in x direction. After the relevant features have been identified, pre-processed data with relevant features are fed into decision tree classifiers to discriminate the data into normal and distracted driving. The results show that the detection accuracy of 78.4% using decision tree can be achieved. By eliminating unhelpful features, the time required to process data is reduced by around 40% to make the proposed technique suitable for real-time application.

Driver distraction detection using machine learning algorithms: an experimental approach

Driver distraction detection using machine learning algorithms: an experimental approach

GSR-based distracted driving identification using discrete & continuous decomposition and wavelet packet transform

Abstract Distracted driving is considered as one of the main factors contributing to the fatalities on the road. Thus, detecting distraction early while driving and alerting the driver can help reduce accidents on the road. While previous studies like as camera-based systems show acceptable detection results, they suffer from distraction detection latency and risk of privacy violation. On the other hand, using physiological signals compensates for early and more secured detection of distraction. Although these technologies achieved high accuracy, they are expensive and extensively intrusive to the drivers. In this study, we propose a minimally intrusive wearable physiological sensor (available on smartwatches) to quantify skin conductance (SC) known as galvanic skin responses (GSR) to characterize and identify distraction during naturalistic driving. 15 driver subjects participated in our experiments, driving normally and then driving while engaged in a secondary task and their GSR data was recorded and labeled accordingly. We employ two deconvolution techniques i) Continuous Decomposition analysis (CDA) and ii) Discrete Decomposition Analysis (DDA) on the raw SC signal to decompose it into the phasic and tonic components. We train various classifiers on the resulted feature matrices in order to identify the data associated with normal driving from the distracted scenarios. Our results show that an averaged 10 fold cross-validation accuracy of 92.2% is achieved using Ensemble bagged classifier on the extracted 74 dimension feature space. The high dimensionality of the original feature space involves high computational cost and might also introduce the curse of dimensionality in the generalization phase. Therefore, we evaluate various types of feature selection methods to eliminate the irrelevant dimensions in the feature space and create a subspace relevant to the task of distraction identification. The improved accuracy of 92.9% and 93.5% is achieved by employing only 10-D and 15-D feature spaces using the embedded random forest feature selection and the ensemble bagged classifier, respectively.

On the Difference Between Necessary and Unnecessary Glances Away From the Forward Roadway: An Occlusion Study on the Motorway.

The present study strove to distinguish traffic-related glances away from the forward roadway from non-traffic-related glances while assessing the minimum amount of visual information intake necessary for safe driving in particular scenarios. Published gaze-based distraction detection algorithms and guidelines for distraction prevention essentially measure the time spent looking away from the forward roadway, without incorporating situation-based attentional requirements. Incorporating situation-based attentional requirements would entail an approach that not only considers the time spent looking elsewhere but also checks whether all necessary information has been sampled. We assess the visual sampling requirements for the forward view based on 25 experienced drivers' self-paced visual occlusion in real motorway traffic, dependent on a combination of situational factors, and compare these with their corresponding glance behavior in baseline driving. Occlusion durations were on average 3 times longer than glances away from the forward roadway, and they varied substantially depending on particular maneuvers and on the proximity of other traffic, showing that interactions with nearby traffic increase perceived uncertainty. The frequency of glances away from the forward roadway was relatively stable across proximity levels and maneuvers, being very similar to what has been found in naturalistic driving. Glances away from the forward roadway proved qualitatively different from occlusions in both their duration and when they occur. Our findings indicate that glancing away from the forward roadway for driving purposes is not the same as glancing away for other purposes, and that neither is necessarily equivalent to distraction.

Classification of Driver Distraction: A Comprehensive Analysis of Feature Generation, Machine Learning, and Input Measures.

The objective of this study was to analyze a set of driver performance and physiological data using advanced machine learning approaches, including feature generation, to determine the best-performing algorithms for detecting driver distraction and predicting the source of distraction. Distracted driving is a causal factor in many vehicle crashes, often resulting in injuries and deaths. As mobile devices and in-vehicle information systems become more prevalent, the ability to detect and mitigate driver distraction becomes more important. This study trained 21 algorithms to identify when drivers were distracted by secondary cognitive and texting tasks. The algorithms included physiological and driving behavioral input processed with a comprehensive feature generation package, Time Series Feature Extraction based on Scalable Hypothesis tests. Results showed that a Random Forest algorithm, trained using only driving behavior measures and excluding driver physiological data, was the highest-performing algorithm for accurately classifying driver distraction. The most important input measures identified were lane offset, speed, and steering, whereas the most important feature types were standard deviation, quantiles, and nonlinear transforms. This work suggests that distraction detection algorithms may be improved by considering ensemble machine learning algorithms that are trained with driving behavior measures and nonstandard features. In addition, the study presents several new indicators of distraction derived from speed and steering measures. Future development of distraction mitigation systems should focus on driver behavior-based algorithms that use complex feature generation techniques.

Head motion coefficient-based algorithm for distracted driving detection

PurposeConcentration is the key to safer driving. Ideally, drivers should focus mainly on front views and side mirrors. Typical distractions are eating, drinking, cell phone use, using and searching things in car as well as looking at something outside the car. In this paper, distracted driving detection algorithm is targeting on nine scenarios nodding, head shaking, moving the head 45° to upper left and back to position, moving the head 45° to lower left and back to position, moving the head 45° to upper right and back to position, moving the head 45° to lower right and back to position, moving the head upward and back to position, head dropping down and blinking as fundamental elements for distracted events. The purpose of this paper is preliminary study these scenarios for the ideal distraction detection, the exact type of distraction.Design/methodology/approachThe system consists of distraction detection module that processes video stream and compute motion coefficient to reinforce identification of distraction conditions of drivers. Motion coefficient of the video frames is computed which follows by the spike detection via statistical filtering.FindingsThe accuracy of head motion analyzer is given as 98.6 percent. With such satisfactory result, it is concluded that the distraction detection using light computation power algorithm is an appropriate direction and further work could be devoted on more scenarios as well as background light intensity and resolution of video frames.Originality/valueThe system aimed at detecting the distraction of the public transport driver. By providing instant response and timely warning, it can lower the road traffic accidents and casualties due to poor physical conditions. A low latency and lightweight head motion detector has been developed for online driver awareness monitoring.

Detection and Evaluation of Driver Distraction Using Machine Learning and Fuzzy Logic

In addition to vehicle control, drivers often perform secondary tasks that impede driving. Reduction of driver distraction is an important challenge for the safety of intelligent transportation systems. In this paper, a methodology for the detection and evaluation of driver distraction while performing secondary tasks is described and an appropriate hardware and a software environment is offered and studied. The system includes a model of normal driving, a subsystem for measuring the errors from the secondary tasks, and a module for total distraction evaluation. A new machine learning algorithm defines driver performance in lane keeping and speed maintenance on a specific road segment. To recognize the errors, a method is proposed, which compares normal driving parameters with ones obtained while conducting a secondary task. To evaluate distraction, an effective fuzzy logic algorithm is used. To verify the proposed approach, a case study with driver-in-the-loop experiments was carried out, in which participants performed the secondary task, namely chatting on a cell phone. The results presented in this research confirm its capability to detect and to precisely measure a level of abnormal driver performance.

Toward Precise Gaze Estimation for Mobile Head-Mounted Gaze Tracking Systems

The gaze estimation in the mobile scenario often suffers from the extrapolation and parallax errors. In this paper, we propose a novel calibration framework to achieve the precise gaze estimation for head-mounted gaze trackers. Our proposed framework consists of two steps to learn a point-to-point and a point-to-line relations, respectively. The aim of step I is to infer the relation between pupil centers and spatially constrained points of regard. By adopting the “CalibMe” gaze data acquisition method, a sparse Gaussian Process using pseudo-inputs is used to capture the smooth residual field unmodeled by the polynomial function. Meanwhile, a distraction detection criterion is introduced to identify the moment when user's attention is taken away from the calibration point thereby removing outliers. By combining with the point-to-point relation inferred in step I, the observed parallax errors are leveraged in step II to obtain a point-to-line relation, i.e., each pupil center will correspond to an epipolar line. Thus, the real image gaze point projected from different depths is predicted as the intersection of two epipolar lines inferred from binocular data. The simulation and experimental results show the effectiveness of our proposed calibration framework for head-mounted gaze trackers.

Real-time detection of driver distraction: random projections for pseudo-inversion-based neural training

There is an accumulating evidence that distracted driving is a leading cause of vehicle crashes and accidents. In order to support safe driving, numerous methods of detecting distraction have been proposed, which are empirically focused on certain driving contexts and gaze behaviour. This paper aims at illustrating a method for the non-intrusive and real-time detection of visual distraction based on vehicle dynamics data and environmental data, without using eye-tracker information. Experiments are carried out in the context of the automotive domain of the European project Holides, which addresses development and qualification of adaptive cooperative human–machine systems, and is co-funded by ARTEMIS Joint Undertaking and Italian University, Educational and Research Department. The collected data are analysed by a single-layer feedforward neural network trained through pseudo-inversion methods, characterized by direct determination of output weights given randomly set input weights and biases. One main feature of our work is the convenient setting of input weights by the so-called sparse random projections: the presence of a great number of null elements in the involved matrices makes especially parsimonious the use at run time of the trained network. Moreover, we use a genetic approach to better explore the input weights network space. The obtained results show better performance with respect to classical pseudo-inversion methods and effective and parsimonious use of memory resources.

Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad hoc methods are often used. In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically weighted ensemble of convolutional neural networks; we show that a weighted ensemble of classifiers using a genetic algorithm yields a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.

The Influence of Different Factors on Right-Turn Distracted Driving Behavior at Intersections Using Naturalistic Driving Study Data

High frequency of distracted driving behavior is considered a high potential risk for traffic safety. Right-turn drivers’ distracted driving behavior can dramatically increase the crash risk considering the complex procedures required by the right-turn movements at intersections. This paper analyzed the influence of several factors including road geometry, environmental factors, and traffic conditions on the occurrence of right-turn drivers’ distracted driving activities. The data were collected through the Naturalistic Driving Study (NDS). A total of 581 events including 208 events with distracted driving and 373 events without distracted driving (baseline events) were extracted from the Strategic Highway Research Program 2 (SHRP 2) NDS database. The logistic model and random forest (RF) were applied for regression analysis. It was found that Vehicle Lane Occupied and Traffic Control are significantly correlated to distracted driving behavior in both models. The analysis of odds ratios indicated that dedicated right-turn lane design and adding yield sign at intersections can reduce the probability of having distracted driving behavior. Traffic density and driving time may also play important roles in the occurrence of distracted driving activities. Countermeasures are recommended to reduce distracted driving in this paper. The findings of this paper can help engineers and researchers better understand the dominant factors affecting drivers’ distraction. This research can also provide theoretical support for the distraction detection function in the advanced driver-assistance systems (ADAS).

Deep Salient Object Detection With Dense Connections and Distraction Diagnosis

In this paper, we propose two novel components for improving deep salient object detection models. The first component, called saliency detection network (S-Net), introduces dense short- and long-range connections that effectively integrate multiscale features to better exploit contexts at multiple levels. Benefiting from the direct access to low- and high-level features, the S-Net can not only exploit the object context but also preserve the object boundary sharply, leading to enhanced saliency detection performance. Second, a distraction detection network (D-Net) is developed to learn to diagnose which regions of an input image are distracting and harmful for saliency prediction of the S-Net. With such distraction diagnosis, the regions that are distracting to S-Net are removed in hindsight from the input image and the resulted distraction-free image is fed to S-Net for saliency prediction. To train the D-Net, a distraction mining approach is proposed to localize the model-specific distracting regions through examining the sensitiveness of the S-Net to image regions in a principled manner. Besides, the distraction mining approach also provides a way to interpret decisions made by deep neural network (DNN) saliency detection models, which relieves the black-box issues of DNNs to some extent. Extensive experiments on seven popular benchmark datasets demonstrate the effectiveness of the combined S-Net and D-Net, which provides new state of the arts.

Detection of Cognitive and Visual Distraction Using Radial Basis Probabilistic Neural Networks

This paper suggests a real-time method for detecting a driver’s cognitive and visual distraction using lateral driving performance measures. The algorithm adopts radial basis probabilistic neural networks (RBPNNs) to construct classification models. In this study, combinations of two driving performance data measures, including the standard deviation of lane position (SDLP) and steering wheel reversal rate (SRR), were considered as measures of distraction. Data for training and testing the RBPNN models were collected under simulated conditions in which fifteen participants drove on a highway. While driving, they were asked to complete auditory recall tasks or arrow search tasks to create cognitively or visually distracted driving periods. As a result, the best performing model could detect distraction with an average accuracy of 78.0 %, which is a relatively high accuracy in the human factors domain. The results demonstrated that the RBPNN model using SDLP and SRR could be an effective distraction detector with easy-to-obtain and inexpensive inputs.

Analysis of Gaze Behavior to Measure Cognitive Distraction in Real-World Driving

Cognitive distraction can impair drivers’ situation awareness and control performance in driving. An on-road study was conducted to examine the efficacy in the detection of driver cognitive distraction based on the driver monitoring system developed by Seeing Machines. Participants completed a 25-km test drive on the local public roads whilst engaging in a series of secondary tasks that were designed to trigger different types of cognitive distraction, such as conversation, comprehension, N-back, and route-planning tasks. The findings showed that percent road center (PRC), one of the promising gaze metrics, increased significantly with cognitive distraction when compared to baseline, but failed to distinguish between different forms of cognitive distraction Moreover, PRC’s sensitivity to cognitive distraction was found to be affected by the chosen radius of road center area. These findings of driver cognitive distraction measurement provide data-driven suggestions for the development of real-time driver monitoring systems in the wild.

Evaluation and Validation of Distraction Detection Algorithms on Multiple Data Sources

Many researchers have developed algorithms to detect distraction, but they have yet to be validated on multiple data sources. This study aims to evaluate these algorithms by comparing their ability to detect distraction and predict event likelihood. Four algorithms that use measures of cumulative glance, past glance behavior, and glance eccentricity were used to understand the distracted state of the driver and were validated on two separate data sources: naturalistic and experimental data. Results showed that there was a higher likelihood of event detection when cumulative glances were considered. Glance eccentricity was best for predicting distraction. Future research can use these findings to design mitigation systems that give drivers feedback in instances of high crash likelihood.

Feature Based Techniques for a Driver's Distraction Detection using Supervised Learning Algorithms based on Fixed Monocular Video Camera

Most of the accidents occur due to drowsiness while driving, avoiding road signs and due to driver's distraction. Driver's distraction depends on various factors which include talking with passengers while driving, mood disorder, nervousness, anger, over-excitement, anxiety, loud music, illness, fatigue and different driver’s head rotations due to change in yaw, pitch and roll angle. The contribution of this paper is two-fold. Firstly, a data set is generated for conducting different experiments on driver’s distraction. Secondly, novel approaches are presented that use features based on facial points; especially the features computed using motion vectors and interpolation to detect a special type of driver’s distraction, i.e., driver’s head rotation due to change in yaw angle. These facial points are detected by Active Shape Model (ASM) and Boosted Regression with Markov Networks (BoRMaN). Various types of classifiers are trained and tested on different frames to decide about a driver's distraction. These approaches are also scale invariant. The results show that the approach that uses the novel ideas of motion vectors and interpolation outperforms other approaches in detection of driver’s head rotation. We are able to achieve a percentage accuracy of 98.45 using Neural Network.

Real‐time detection of distracted driving based on deep learning

Driver distraction is a leading factor in car crashes. With a goal to reduce traffic accidents and improve transportation safety, this study proposes a driver distraction detection system which identifies various types of distractions through a camera observing the driver. An assisted driving testbed is developed for the purpose of creating realistic driving experiences and validating the distraction detection algorithms. The authors collected a dataset which consists of images of the drivers in both normal and distracted driving postures. Four deep convolutional neural networks including VGG-16, AlexNet, GoogleNet, and residual network are implemented and evaluated on an embedded graphic processing unit platform. In addition, they developed a conversational warning system that alerts the driver in real-time when he/she does not focus on the driving task. Experimental results show that the proposed approach outperforms the baseline one which has only 256 neurons in the fully-connected layers. Furthermore, the results indicate that the GoogleNet is the best model out of the four for distraction detection in the driving simulator testbed.

Visual-Manual Distraction Detection Using Driving Performance Indicators With Naturalistic Driving Data

This paper investigates the problem of driver distraction detection using driving performance indicators from onboard kinematic measurements. First, naturalistic driving data from the integrated vehicle-based safety system program are processed, and cabin camera data are manually inspected to determine the driver’s state (i.e., distracted or attentive). Second, existing driving performance metrics, such as steering entropy, steering wheel reversal rate, and lane offset variance, are reviewed against the processed naturalistic driving data. Furthermore, a nonlinear autoregressive exogenous (NARX) driving model is developed to predict vehicle speed based on the range (distance headway), range rate, and speed history. For each driver, the NARX model is then trained on the attentive driving data. We show that the prediction error is correlated with driver distraction. Finally, two features, steering entropy and mean absolute speed prediction error from the NARX model are selected, and a support vector machine is trained to detect driving distraction. Prediction performances are reported.

Real-time detection of drivers’ texting and eating behavior based on vehicle dynamics

With rapid advancement in cellphones and intelligent in-vehicle technologies along with driver’s inclination to multitasking, crashes due to distracted driving had become a growing safety concern in our road network. Some previous studies attempted to detect distracted driving behaviors in real-time to mitigate their adverse consequences. However, these studies mainly focused on detecting either visual or cognitive distractions only, while most of the real-life distracting tasks involve driver’s visual, cognitive, and physical workload, simultaneously. Additionally, previous studies frequently used eye, head, or face tracking data, although current vehicles are not commonly equipped with technologies to acquire such data. Also those data are comparatively difficult to acquire in real-time during traffic monitoring operations. To address the above issues, this study focused on developing algorithms for detecting distraction tasks that involve simultaneous visual, cognitive, and physical workload using only vehicle dynamics data. Specifically, algorithms were developed to detect driving behaviors under two distraction tasks – texting and eating. Experiment was designed to include the two distracted driving scenarios and a control with multiple runs for each. A medium fidelity driving simulator was used for acquiring vehicle dynamics data for each scenario and each run. Several data mining techniques were explored in this study to investigate their performance in detecting distraction. Among them, the performance of two linear (linear discriminant analysis and logistic regression) and two nonlinear models (support vector machines and random forests) is reported in this article. Random forests algorithms had the best performance, which detected texting and eating distraction with an accuracy of 85.38% and 81.26%, respectively. This study may provide useful guidance to successful development and implementation of distracted driver detection algorithms in connected vehicle environment, as well as to auto manufacturers interested in integrating distraction detection systems in their vehicles.