Distraction Detection Research Articles

Online distraction detection for naturalistic driving dataset using kinematic motion models and a multiple model algorithm

Detecting distracted driving is important for developing Advanced Driver Assistance Systems and improving road safety. Most of the existing research analyzes drivers directly via video analysis techniques or by measuring cognitive load, however these approaches often require additional sensors to be installed in vehicles or equipped to drivers. Given that most distractions may have a direct influence on drivers’ control of vehicles, this paper proposes a new method to utilize available vehicle kinematic data for detecting distracted driving. The proposed method predicts vehicle kinematics by fusing multiple state–space models that capture different driving motion patterns under normal driving. An online monitoring scheme is developed by using Exponentially Weighted Moving Average (EWMA) and Cumulative Sum (CUSUM) charts, which detects abnormal mean shifts of lateral speeds and prediction errors of lane positions to provide warnings of distracted driving. A case study is presented based on two naturalistic driving datasets — the Integrated Vehicle-Based Safety Systems (IVBSS) and Safety Pilot Model Deployment (SPMD) datasets.

Optimally-Weighted Image-Pose Approach (OWIPA) for Distracted Driver Detection and Classification

Distracted driving is the prime factor of motor vehicle accidents. Current studies on distraction detection focus on improving distraction detection performance through various techniques, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, the research on detection of distracted drivers through pose estimation is scarce. This work introduces an ensemble of ResNets, which is named Optimally-weighted Image-Pose Approach (OWIPA), to classify the distraction through original and pose estimation images. The pose estimation images are generated from HRNet and ResNet. We use ResNet101 and ResNet50 to classify the original images and the pose estimation images, respectively. An optimum weight is determined through grid search method, and the predictions from both models are weighted through this parameter. The experimental results show that our proposed approach achieves 94.28% accuracy on AUC Distracted Driver Dataset.

Dimensionally improved residual neural network to detect driver distraction in real time

Bountiful reasons are there for crop up of road accidents. Predominantly most of the road accidents are turn out by the humans. In compare with other sources of road accidents, distracted driving is the foremost one. Driver gets distracted ascribed to many factors such as texting using phone, seeing outside of the car. The regime can grasp the road accidents by enforcing the traffic regulations moreover the regime seeks the assistance from the technology to reduce the car accidents due to driver distraction. There are numerous research undergoing to reduce the chance of the driver to get distracted by monitoring the physical activities of the driver while driving the four wheeler. Automatic detection of driver distraction can support to develop an apprise system by observing the activities of the driver that provide better result and evading the car accidents. Deep learning is a leading technology in automated industries to detect the traffic light signs and pedestrians. The novel ResNext 101 is the proposed work and inherited from its parent model called Residual Network (ResNet) to train and test the model. The “Next” in the word ResNext represents next dimension of ResNet called as “cardinality”. In this paper, with the help of Convolutional Neural Network (CNN), the system detects and captures the distracted moments of the driver from proper driving gestures automatically in the form of 2D image. The test results of the model outperforms and proved its significance over the existing driver detection algorithms by bagging higher efficacy with 97.6% accuracy in classifying the live driver posture.

High-wearable EEG-based distraction detection in motor rehabilitation

A method for EEG-based distraction detection during motor-rehabilitation tasks is proposed. A wireless cap guarantees very high wearability with dry electrodes and a low number of channels. Experimental validation is performed on a dataset from 17 volunteers. Different feature extractions from spatial, temporal, and frequency domain and classification strategies were evaluated. The performances of five supervised classifiers in discriminating between attention on pure movement and with distractors were compared. A k-Nearest Neighbors classifier achieved an accuracy of 92.8 ± 1.6%. In this last case, the feature extraction is based on a custom 12 pass-band Filter-Bank (FB) and the Common Spatial Pattern (CSP) algorithm. In particular, the mean Recall of classification (percentage of true positive in distraction detection) is higher than 92% and allows the therapist or an automated system to know when to stimulate the patient’s attention for enhancing the therapy effectiveness.

Towards a Context-Dependent Multi-Buffer Driver Distraction Detection Algorithm

This paper presents initial work on a context-dependent driver distraction detection algorithm called AttenD2.0, which extends the original AttenD algorithm with elements from the Minimum Required Attention (MiRA) theory. Central to the original AttenD algorithm is a time buffer which keeps track of how often and for how long the driver looks away from the forward roadway. When the driver looks away the buffer is depleted and when looking back the buffer fills up. If the buffer runs empty the driver is classified as distracted. AttenD2.0 extends this concept by adding multiple buffers, thus integrating situation dependence and visual time-sharing behaviour in a transparent manner. Also, the increment and decrement of the buffers are now controlled by both static requirements (e.g. the presence of an on-ramp increases the need to monitor the sides and the mirrors) as well as dynamic requirements (e.g., reduced speed lowers the need to monitor the speedometer). The algorithm description is generic, but a real-time implementation with concrete values for different parameters is showcased in a driving simulator experiment with 16 bus drivers, where AttenD2.0 was used to ensure that drivers are attentive before taking back control after an automated bus stop docking and depot procedure. The scalability of AttenD2.0 relative to available data sources and the level of vehicle automation is demonstrated. Future work includes expanding the concept to real-world environments by automatically integrating situational information from the vehicles environmental sensing and from digital maps.

Driver Distraction Detection and Early Prediction and avoidance of accidents using Convolutional Neural Networks

In this fast-moving world, accidents in four wheeled vehicles occur due to the break failure or because of the carelessness or the fatigue of the driver. The driver pattern of the driver plays a major role in providing road safety as well as in fuel consumption. The distraction of drivers is found by installing various sensors which is used for gathering real time data. The behaviour of drivers under stress condition and their behavioural patterns for early detection and avoidance of accidents are found using convolutional neural networks. Convolutional Neural Networks are efficient classifiers in handling image processing and computer vision problem. The input dataset is a collection of driving behaviour of 10 different drivers collected from Kaggle. The behaviour of drivers under 7 distracted situations like texting, talking through phone, playing music, drinking, eating, doing make up and talking to passenger are considered. The batch normalization is used at the right of the input layer in order to avoid skewing of data at a direction. It is shown, the convolutional neural networks at 4 epochs have shown 99% accuracy.

Monitoring distraction of construction workers caused by noise using a wearable Electroencephalography (EEG) device

In the construction environment with high attention requirements, distraction is the main cause of unsafe behavior and safety performance degradation. However, few studies have focused on distraction's cognitive features and how to monitor it objectively in the construction workplace. To fill the research gap, the present study examined the correlation between distraction and brain activity using an Electroencephalography (EEG) device, intending to provide an approach for objectively monitoring worker distraction. In the simulated hazards identification activity, sustained attention to response task and dual-task paradigms have been employed to induce distraction combined with noise interference. Twenty-seven subjects participated in the experiment to identify whether a hazardous opening exists or not in the workplace in the shown images. The EEG waves were recorded and divided into two groups according to task performance: focused and distracted. Through feature calculation and extraction, it was found that beta and gamma powers in the left temporal and right pre-frontal cortex can distinguish these two statuses, particularly in channels T7 and AF4. The indicators can be considered as an objective evaluation of an individual's sustained attention and attention failures. The developed indicators located in specified brain zones can also be used as a reference for attention training. By providing safety managers with attention status about the workers in high-risk workplaces, distraction detection contributes to control and regulate work error and improper operation, which can extend to apply in other attentive jobs like drivers, pilots, surgeons, and lifeguards.

Vision Based Detection of Driver Cell Phone Usage and Food Consumption

Distracted driving is a problem which yearly causes a large amount of road traffic crashes with high rates of fatalities and injured persons. Recently, car manufacturers started to integrate driver monitoring systems to detect visual distraction. This paper proposes a method to extend such systems by driver posture classification to detect driver cell phone usage and food consumption. Such an extension can be beneficial since systems that focus on the detection of visual distraction mainly rely on head pose and gaze information. Thus, distraction caused by cell phone usage or food consumption can not be detected by these systems when the driver is looking to the road ahead. To robustly detect those types of manual and cognitive distraction, different deep learning models were trained and evaluated based on a new image dataset which was captured by two infrared cameras to ensure that a large range of head angles can be covered by the system. Separate Convolutional Neural Networks (CNNs) were trained and evaluated for the dataset of the left and the right camera to optimize the classification accuracy. The trained CNNs revealed a competitive test accuracy of 92.88% and 90.36% for the left and the right camera, respectively. In inference mode, the models achieve a frame rate of 44Hz and 28Hz for the left and the right camera, respectively. The combination of the classification output of both networks revealed a test accuracy of 92.54%.

Driver Distraction Detection Methods: A Literature Review and Framework

Driver inattention and distraction are the main causes of road accidents, many of which result in fatalities. To reduce road accidents, the development of information systems to detect driver inattention and distraction is essential. Currently, distraction detection systems for road vehicles are not yet widely available or are limited to specific causes of driver inattention such as driver fatigue. Despite the increasing automation of driving due to the availability of increasingly sophisticated assistance systems, the human driver will continue to play a longer role as supervisor of vehicle automation. With this in mind, we review the published scientific literature on driver distraction detection methods and integrate the identified approaches into a holistic framework that is the main contribution of the paper. Based on published scientific work, our driver distraction detection framework contains a structured summary of reviewed approaches for detecting the three main distraction detection approaches: manual distraction, visual distraction, and cognitive distraction. Our framework visualizes the whole detection information chain from used sensors, measured data, computed data, computed events, inferred behavior, and inferred distraction type. Besides providing a sound summary for researchers interested in distracted driving, we discuss several practical implications for the development of driver distraction detection systems that can also combine different approaches for higher detection quality. We think our research can be useful despite - or even because of - the great developments in automated driving.

Distracted driver detection using compressed energy efficient convolutional neural network

The availability of techniques for driver distraction detection has been difficult to put to use because of delays caused due to lag in inferencing the model. Distractions caused due to handheld devices have been major causes of traffic accidents as they affect the decision-making capabilities of the driver and gives them less time to react to difficult situations. Often drivers try to multitask which reduces their reaction time leading to accidents, which can easily be avoided if they had been attentive. As such, problems related to the driver’s negligence towards safety a possible solution is to monitor the driver and driving behavior and alerting them if they are distracted. In this paper, we propose a novel approach for detecting when a driver is distracted due to in hand electronic devices which is not only able to detect the distraction with high accuracy but also is energy and memory efficient. Our proposed compressed neural got an accuracy of 0.83 in comparison to 0.86 of heavyweight network.

Robust Deep Learning-Based Driver Distraction Detection and Classification

Robust Deep Learning-Based Driver Distraction Detection and Classification

Современные подходы к детектированию отвлечения внимания водителя в кабине транспортного средства

In this paper the author reviews various methods for detecting driver distractions. The considered methods are structured and classifi ed according to such characteristics as the possibility of implementation in the car, used sources of data, used algorithms, and the possibility of applying them on the fl y. Distraction detection allows to prevent traffi c accidents, and the collected statistics can be used as one of the factors for calculation of driver ratings.

How Do Drivers Hold Their Phone? Age, Prevalence, & Handedness

Objective Research shows frequent mobile phone use in vehicles but says little regarding how drivers hold their phone. This knowledge would inform countermeasures and benefit law enforcement in detecting phone use. Methods 934 participants were surveyed over phone-use prevalence, handedness, traffic-direction, and where they held their device. Results The majority (66%) reported using their phone while driving. Younger drivers were more likely to use their device. Of device-users, 67% preferred their passenger-side hand, 25% driver-side, and 8% both. Height- wise: 22% held in-lap, 52% even with the wheel, and 22% at wheel-top. Older drivers were more likely to hold the phone in the highest position The three most popular combinations were passenger-middle (35%), passenger-low (19%), and passenger-high (13.9%). There was insufficient evidence of differences based on handedness, prevalence, or traffic-direction. Conclusion Driver-preferred attention regions often require substantial neck flexion and eye-movement, which facilitates distraction detection. However, behavior may change in response to future interventions.

Driver Distraction Detection Method Based on Continuous Head Pose Estimation

In view of the fact that the detection of driver’s distraction is a burning issue, this study chooses the driver’s head pose as the evaluation parameter for driving distraction and proposes a driver distraction method based on the head pose. The effects of single regression and classification combined with regression are compared in terms of accuracy, and four kinds of classical networks are improved and trained using 300W-LP and AFLW datasets. The HPE_Resnet50 with the best accuracy is selected as the head pose estimator and applied to the ten-category distracted driving dataset SF3D to obtain 20,000 sets of head pose data. The differences between classes are discussed qualitatively and quantitatively. The analysis of variance shows that there is a statistically significant difference in head posture between safe driving and all kinds of distracted driving at 95% and 90% confidence levels, and the postures of all kinds of driving movements are distributed in a specific Euler angle range, which provides a characteristic basis for the design of subsequent recognition methods. In addition, according to the continuity of human movement, this paper also selects 90 drivers’ videos to analyze the difference in head pose between safe driving and distracted driving frame by frame. By calculating the spatial distance and sample statistics, the results provide the reference point, spatial range, and threshold of safe driving under this driving condition. Experimental results show that the average error of HPE_Resnet50 in AFLW2000 is 6.17° and that there is an average difference of 12.4° to 54.9° in the Euler angle between safe driving and nine kinds of distracted driving on SF3D.

Evaluating Driver Features for Cognitive Distraction Detection and Validation in Manual and Level 2 Automated Driving.

This study aimed to investigate the impacts of feature selection on driver cognitive distraction (CD) detection and validation in real-world nonautomated and Level 2 automated driving scenarios. Real-time driver state monitoring is critical to promote road user safety. Twenty-four participants were recruited to drive a Tesla Model S in manual and Autopilot modes on the highway while engaging in the N-back task. In each driving mode, CD was classified by the random forest algorithm built on three "hand-crafted" glance features (i.e., percent road center [PRC], the standard deviation of gaze pitch, and yaw angles), or through a large number of features that were transformed from the output of a driver monitoring system (DMS) and other sensing systems. In manual driving, the small set of glance features was as effective as the large set of machine-generated features in terms of classification accuracy. Whereas in Level 2 automated driving, both glance and vehicle features were less sensitive to CD. The glance features also revealed that the misclassified driver state was the result of the dynamic fluctuations and individual differences of cognitive loads under CD. Glance metrics are critical for the detection and validation of CD in on-road driving. The paper suggests the practical value of human factors domain knowledge in feature selection and ground truth validation for the development of driver monitoring technologies.

Deep unsupervised multi-modal fusion network for detecting driver distraction

The risk of incurring a road traffic crash has increased year by year. Studies show that lack of attention during driving is one of the major causes of traffic accidents. In this work, in order to detect driver distraction, e.g., phone conversation, eating, texting, we introduce a deep unsupervised multi-modal fusion network, termed UMMFN. It is an end-to-end model composing of three main modules: multi-modal representation learning, multi-scale feature fusion and unsupervised driver distraction detection. The first module is to learn low-dimensional representation of multiple heterogeneous sensors using embedding subnetworks. The goal of multi-scale feature fusion is to learn both the temporal dependency for each modality and spatio dependencies from different modalities. The last module utilizes a ConvLSTM Encoder-Decoder model to perform an unsupervised classification task that is not affected by new types of driver behaviors. During the detection phase, a fine-grained detection decision can be made through calculating reconstruction error of UMMFN as a score for each captured testing data. We empirically compare the proposed approach with several state-of-the-art methods on our own multi-modal dataset for distracted driving behavior. Experimental results show that UMMFN has superior performance over the existing approaches.

A deep learning-based driver distraction identification framework over edge cloud

Currently, the number of traffic accidents has been increased globally. One of the main reasons for this increase is the distraction of the driver on the road. Distracted driving can cause collisions and cause injury, death, or property damage. New techniques can help to mitigate this problem, and one of the recent approaches is to employ body wearable sensors or camera sensors in the vehicle for real-time monitoring and detection of drivers’ distraction and behaviors, such as cell phone use, talking, eating, drinking, radio tuning, navigation interaction, or even combing hair while driving. However, this type of approach requires not only a powerful training module but also a lightweight module for real-time detection and analyzing the captured data. Data need to be collected from specific wearable or camera sensors in order to detect drivers’ distraction and ensure immediate feedback by the administrator for safe driving. Therefore, in this paper, we propose an effective camera-based framework for real-time identification of drivers’ distraction by using a deep learning approach with edge and cloud computing technologies. More specifically, the framework consists of three modules, including the distraction detection module deployed on edge devices in the vehicle environment, the training module deployed in the cloud environment, and finally the analyzing module implemented in the monitoring environment (administrator side) connected with a telecommunication network. The proposed framework is developed using two deep learning models. The first is a custom deep convolutional neural network (CDCNN) model, and the second one is a visual geometry group-16 (VGG16)-based fine-tuned model. Several experiments are conducted on a public large-scale driver distraction dataset to evaluate the two models. The experimental results show that the accuracy rates were 99.64% for the first model and 99.73% for the second model using a holdout test set of 10%. In addition, the first and second models have achieved accuracy rates of 99.36% and 99.57% using a holdout test set of 30%. The results confirmed the applicability and appropriateness of the adopted deep learning models for designing the proposed driver distraction detection framework.

Support vector machines for the identification of real-time driving distraction using in-vehicle information systems

IVIS (In-vehicle Information System) is an important factor causing driver distraction. To study the driver distraction detection method when operating IVIS, the effectiveness of driving performance indicators in the identification of driving distraction was verified by the method of variance analysis. Forty participants were selected to conduct the driver distraction experiment when operating IVIS, and the data of driving performance indicators such as eye movement, speed, et al. were obtained. According to the driving performance data of IVIS, a real-time detection of distraction based on driving performance was built by using support vector machine, and three kernel functions of the model were conducted comparative analysis and validation. The results show that SVM models can effectively evaluate the degree of drivers’ distraction. At the same time, when the Radial Basis Function is used as a kernel function, the accuracy for recognizing driver distraction is 89.9%, which is higher than when the sigmoid polynomial kernel function and SAVE-IT model are used. The research could be applied in the design of adaptive in-vehicle systems and the evaluation of driving distraction, providing theoretical support and reference for the development of vehicle-mounted information systems and the management of driver distraction prevention measures. Highlights We verify the effectiveness of driving performance indicators as decision variables for driving distraction. We use driving performance indicators as inputs to a distraction detection algorithm. We examine changes in the level of information about driving distraction. SVMs method can be used to detect driving distraction during IVIS operations in real time. The accuracy for using Radial Basis Function to recognizing the driver’s level of distraction is higher than the Radial Basis Function.

Human Distraction Detection from Video Stream Using Artificial Emotional Intelligence

Human Distraction Detection from Video Stream Using Artificial Emotional Intelligence

Detection of driver manual distraction via image-based hand and ear recognition

Driving distraction is a leading cause of fatal car accidents, and almost nine people are killed in the US each day because of distracting activities. Therefore, reducing the number of distraction-affected traffic accidents remains an imperative issue. A novel algorithm for detection of drivers’ manual distraction was proposed in this manuscript. The detection algorithm consists of two modules. The first module predicts the bounding boxes of the driver's right hand and right ear from RGB images. The second module takes the bounding boxes as input and predicts the type of distraction. 106,677 frames extracted from videos, which were collected from twenty participants in a driving simulator, were used for training (50%) and testing (50%). For distraction classification, the results indicated that the proposed framework could detect normal driving, using the touchscreen, and talking with a phone with F1-score 0.84, 0.69, 0.82, respectively. For overall distraction detection, it achieved F1-score of 0.74. The whole framework ran at 28 frames per second. The algorithm achieved comparable overall accuracy with similar research, and was more efficient than other methods. A demo video for the algorithm can be found at https://youtu.be/NKclK1bHRd4.