Driver Drowsiness Detection Research Articles

A novel deep-learning model based on τ-shaped convolutional network (τNet) with long short-term memory (LSTM) for physiological fatigue detection from EEG and EOG signals.

In recent years, fatigue driving has become the main cause of traffic accidents, leading to increased attention towards fatigue detection systems. However, the pooling and strided convolutional operations in fatigue detection algorithm based on traditional deep learning methods may led to the loss of some useful information. This paper proposed a novel -shaped convolutional network ( ) aiming to address this issue. Unlike traditional network structures, incorporates the operations of upsampling features and concatenating high- and low-level features, enabling full utilization of useful information. Moreover, considering that the fatigue state is a mental state involving temporal evolution, we proposed the novel long short-term memory (LSTM)- -shaped convolutional network (LSTM- ), a parallel structure composed of LSTM and for fatigue detection, where extracts time-invariant features with location information, and LSTM extracts long temporal dependencies. We compared LSTM- with six competing methods based on two datasets. Results showed that the proposed algorithm achieved higher classification accuracy than the other methods, with 94.25% on EEG data (binary classification) and 82.19% on EOG data (triple classification). Additionally, the proposed algorithm exhibits low computational cost, good training stability, and robustness against insufficient training. Therefore, it is promising for further implementation of fatigue online detection systems.

A generative adaptive convolutional neural network with attention mechanism for driver fatigue detection with class-imbalanced and insufficient data

Over the past few years, fatigue driving has emerged as one of the main causes of traffic accidents, necessitating the development of driver fatigue detection systems. However, many existing methods involves tedious manual parameter tunings, a process that is both time-consuming and results in task-specific models. On the other hand, most of the researches on fatigue recognition are based on class-balanced and sufficient data, and effectively “mine” meaningful information from class-imbalanced and insufficient data for fatigue recognition is still a challenge. In this paper, we proposed two novel models, the attention-based residual adaptive multiscale fully convolutional network-long short term memory network (ARMFCN-LSTM), and the Generative ARMFCN-LSTM (GARMFCN-LSTM) aiming to address this issue. ARMFCN-LSTM excels at automatically extracting multiscale representations through adaptive multiscale temporal convolutions, while capturing temporal dependency features through LSTM. GARMFCN-LSTM integrates Wasserstein GAN with gradient penalty (WGAN-GP) into ARMFCN-LSTM to improve driver fatigue detection performance by alleviating data scarcity and addressing class imbalances. Experimental results show that ARMFCN-LSTM achieves the highest classification accuracy of 95.84% in driver fatigue detection on the class-balanced EEG dataset (binary classification), and GARMFCN-LSTM attained an improved classification accuracy of 84.70% on the class-imbalanced EOG dataset (triple classification), surpassing the competing methods. Therefore, the proposed models are promising for further implementations in online driver fatigue detection systems.

Preventive Detection of Driver Drowsiness from EEG Signals using Fuzzy Expert Systems

Currently, the percentage of traffic accidents has increased, and according to statistics, this percentage will continue to increase every year, so it is necessary to develop new technologies to prevent this kind of accidents. This paper presents a drowsiness detection system based on electroencephalogram (EEG) signals using a pair of channels (Fp1 and Fp2) applied to drivers before entering their vehicles. First, this model detects the relationship between the area under the curve (AUC) of alpha brain waves, an effective parameter for detecting drowsiness. Then, the extracted information is passed to a fuzzy expert system (FES) that classifies the subject's state as "alert" or "sleepy"; the criterion used was a threshold and training with subjective levels. The proposed system was compared with neural network models, such as support vector machine (SVM), K nearest neighbors (KNN), and random forest (RF). Measurements of one hundred and twenty minutes were performed on each of the ten drivers for two days to test the system. The tests confirm that this system is suitable for preventive measures and that the fuzzy system is superior to traditional neural network methods.

Enhanced performance of EEG-based brain-computer interfaces by joint sample and feature importance assessment.

Electroencephalograph (EEG) has been a reliable data source for building brain-computer interface (BCI) systems; however, it is not reasonable to use the feature vector extracted from multiple EEG channels and frequency bands to perform recognition directly due to the two deficiencies. One is that EEG data is weak and non-stationary, which easily causes different EEG samples to have different quality. The other is that different feature dimensions corresponding to different brain regions and frequency bands have different correlations to a certain mental task, which is not sufficiently investigated. To this end, a Joint Sample and Feature importance Assessment (JSFA) model was proposed to simultaneously explore the different impacts of EEG samples and features in mental state recognition, in which the former is based on the self-paced learning technique while the latter is completed by the feature self-weighting technique. The efficacy of JSFA is extensively evaluated on two EEG data sets, i.e., SEED-IV and SEED-VIG. One is a classification task for emotion recognition and the other is a regression task for driving fatigue detection. Experimental results demonstrate that JSFA can effectively identify the importance of different EEG samples and features, leading to enhanced recognition performance of corresponding BCI systems.

Modulating Driver Alertness via Ambient Olfactory Stimulation: A Wearable Electroencephalography Study.

Poor alertness levels and related changes in cognitive efficiency are common when performing monotonous tasks such as extended driving. Recent studies have investigated driver alertness decrement and possible strategies for modulating alertness with the goal of improving reaction times to safety critical events. However, most studies rely on subjective measures in assessing alertness changes, while the use of olfactory stimuli, which are known to be strong modulators of cognitive states, has not been commensurately explored in driving alertness settings. To address this gap, in the present study we investigated the effectiveness of olfactory stimuli in modulating the alertness state of drivers and explored the utility of electroencephalography (EEG) in developing objective brain-based tools for assessing the resulting changes in cortical activity. Olfactory stimulation induced a significant differential effect on braking reaction time. The corresponding effect to the cortical activity was characterized using EEG-derived metrics and the devised machine learning framework yielded a high discriminating accuracy (92.1%). Furthermore, neural activity in the alpha frequency band was found to be significantly associated with the observed drivers' behavioral changes. Overall, our results demonstrate the potential of olfactory stimuli to modulate the alertness state and the efficiency of EEG in objectively assessing the resulting cognitive changes.

Enhancing Road Safety: Deep Learning-Based Intelligent Driver Drowsiness Detection for Advanced Driver-Assistance Systems

Driver drowsiness detection is a significant element of Advanced Driver-Assistance Systems (ADASs), which utilize deep learning (DL) methods to improve road safety. A driver drowsiness detection system can trigger timely alerts like auditory or visual warnings, thereby stimulating drivers to take corrective measures and ultimately avoiding possible accidents caused by impaired driving. This study presents a Deep Learning-based Intelligent Driver Drowsiness Detection for Advanced Driver-Assistance Systems (DLID3-ADAS) technique. The DLID3-ADAS technique aims to enhance road safety via the detection of drowsiness among drivers. Using the DLID3-ADAS technique, complex features from images are derived through the use of the ShuffleNet approach. Moreover, the Northern Goshawk Optimization (NGO) algorithm is exploited for the selection of optimum hyperparameters for the ShuffleNet model. Lastly, an extreme learning machine (ELM) model is used to properly detect and classify the drowsiness states of drivers. The extensive set of experiments conducted based on the Yawdd driver database showed that the DLID3-ADAS technique achieves a higher performance compared to existing models, with a maximum accuracy of 97.05% and minimum computational time of 0.60 s.

Eye Blink Detection- A New System for Driver Drowsiness and Distraction Detection

The advancement of technology has led to innovative solutions for ensuring road safety, particularly in the detection of driver drowsiness and distraction. This abstract presents a novel approach using eye blink detection as a fundamental component in a system designed to mitigate the risks associated with drowsy and distracted driving. The proposed system integrates cutting-edge computer vision techniques to accurately detect and analyze eye blinks in real-time. By utilizing high-resolution cameras installed within the vehicle, the system captures and processes images of the driver's face, focusing on the eyes to track blinking patterns and durations. Through sophisticated algorithms and machine learning models, it distinguishes between normal blinks and extended periods of eye closure, indicative of drowsiness or distraction. Furthermore, the system incorporates contextual awareness by considering various factors such as environmental conditions, driving behavior, and facial expressions to enhance the accuracy of detection. Machine learning algorithms trained on diverse datasets enable the system to adapt and improve its performance over time, making it more adept at recognizing subtle variations in blink patterns unique to each driver. The applications of this system are multifaceted. Firstly, it serves as an early warning system, alerting drivers when signs of drowsiness or distraction are detected. This proactive approach aims to prevent accidents by prompting drivers to take necessary breaks or refocus their attention on the road. Moreover, the system can integrate with existing driver assistance technologies, allowing for automated interventions, such as adjusting seat vibrations or emitting audible alerts to re-engage the driver. In conclusion, the proposed eye blink detection system represents a significant advancement in the realm of driver safety technologies. By leveraging sophisticated algorithms and real-time analysis of blink patterns, it offers a promising solution to mitigate the risks associated with driver drowsiness and distraction, ultimately contributing to safer roads and reducing the likelihood of accidents. KEYWORDS: Eye blinking, Detection, Driver, Drowsiness, Distraction, Detection, computer vision, machine learning, Deep learning, image processing.

Utilizing commercial off-the-shelf smartwatches in a real-time drowsiness detection system

Utilizing commercial off-the-shelf smartwatches in a real-time drowsiness detection system

Driver Drowsiness Detection using CNN

Abstract: This research introduces a Driver Drowsiness Detection system employing Convolutional Neural Networks (CNN). The system analyses real-time facial features from in-vehicle cameras to determine a driver's alertness level. Trained on diverse datasets, the CNN model demonstrates high accuracy in identifying drowsiness signs, making it suitable for real-world deployment. This system contributes to road safety by providing timely alerts to prevent accidents caused by driver fatigue. As road safety remains a critical concern, the development of intelligent systems to mitigate driver-related risks has become imperative. Driver drowsiness is a major factor contributing to road accidents, emphasizing the need for robust and real-time detection mechanisms. This research presents a novel approach for Driver Drowsiness Detection using Convolutional Neural Networks (CNN). The proposed system utilizes CNN architecture to analyse facial features extracted from real-time video streams captured by an in-vehicle camera. Facial landmarks and expressions are processed to determine the driver's level of alertness. The CNN model is trained on a diverse dataset comprising both drowsy and alert facial expressions, ensuring its adaptability to different driving conditions and individual characteristics. The system's effectiveness is evaluated through extensive experiments using various datasets and scenarios. The results demonstrate the CNN's capability to accurately identify signs of driver drowsiness, achieving high precision and recall rates. Furthermore, the model's real-time processing capabilities make it suitable for deployment in practical, on-road scenarios. The proposed Driver Drowsiness Detection system presents a promising solution to enhance road safety by providing timely alerts to drivers when signs of drowsiness are detected. This research contributes to the ongoing efforts to integrate artificial intelligence into vehicle safety systems, ultimately reducing the risk of accidents caused by driver fatigue.

A review of techniques and methods for deep learning techniques in driver fatigue detection

Road accidents in which fatigue driving is a significant cause of death are responsible for many deaths worldwide. Approximately 100,000 crashes are caused by driver fatigue each year. Also, fatigue driving is responsible for about 16% of road accidents in general and more than 20% of highway accidents, so fatigue driving accounts for a large percentage of vehicle accidents. Fatigue driving detection usually uses subjective and objective methods. Subjective methods rely on analysing the driver's psychological and facial expression information, while objective methods use external devices to extract feature parameters and apply artificial intelligence algorithms. However, these methods have limitations, such as subjectivity and individual differences. Deep learning, a promising tool inspired by neural networks, offers automatic feature learning, robust pattern recognition, and high adaptability. This review explores the application of deep learning in fatigue driving detection. It examines various deep learning feature extraction methods, classification models, prediction models, and related datasets. By leveraging deep learning techniques, fatigue driving detection can achieve higher accuracy and effectiveness, providing a reliable solution to this critical road safety problem. The review concludes with recommendations and future perspectives in this area.

Study on Fatigue Driving Detection based on Physiological Characteristics of Drivers

Li, S. (2022). Best Practice to Calculate and Interpret Model Feature Importance: With an example of Random Forest model. Published in Towards Data Science. https://towardsdatascience.com/best-practice-to-calculate-and-interpret-model-feature-importance-14f0e11ee660.

Advanced Driver-Assistance Systems: Features Journey for Tomorrow

Intelligent associated vehicles (ICVs) are accepted to completely change people sooner rather than later by making the transportation more secure, cleaner and more agreeable. Albeit numerous models of ICVs have been created to demonstrate the idea of independent driving and the plausibility of further developing traffic effectiveness, there actually exists a critical hole prior to accomplishing large scale manufacturing of undeniable level ICVs. The goal of this study is to introduce an outline of both the cutting edge and future viewpoints of key necessary advances for future ICVs. It is a moving undertaking to survey every connected work and foresee their future viewpoints, particularly for such a perplexing and interdisciplinary area of examination. Advanced driver-assistance systems (ADASs) have become a salient feature for safety in modern vehicles. They are also a key underlying technology in emerging autonomous vehicles. State-of-the-art ADASs are primarily vision based, various type of features for ex. Lane Departure Warning (LDW), Blind Spot Monitoring (BSM), Forward Collision Warning (FCW), Automatic Emergency Braking (AEB), Traffic Sign Recognition, High Beam Assist, Rear Cross Traffic Alert, Driver Drowsiness Detection, Obstacle Aware Acceleration, Auto-steer etc. The paper aims at giving a complete picture focusing on the ADAS features for the user-friendly design of human-machine interfaces between driver and assistance system.

Impact and Classification of Body Stature and Physiological Variability in the Acquisition of Vital Signs Using Continuous Wave Radar

The Bio-Radar system, useful for monitoring patients with infectious diseases and detecting driver drowsiness, has gained popularity in the literature. However, its efficiency across diverse populations considering physiological and body stature variations needs further exploration. This work addresses this gap by applying machine learning (ML) algorithms—Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Random Forest—to classify subjects based on gender, age, Body Mass Index (BMI), and Chest Wall Perimeter (CWP). Vital signs were collected from 92 subjects using a Continuous Wave (CW) radar operating at 5.8 GHz. The results showed that the Random Forest algorithm was the most accurate, achieving accuracies of 76.66% for gender, 71.13% for age, 72.52% for BMI, and 74.61% for CWP. This study underscores the importance of considering individual variations when using Bio-Radar, enhancing its efficiency and expanding its potential applications.

Driver Drowsiness Detection using Machine Learning Approach

Driver fatigue is a major cause of traffic accidents. The number of deaths and injuries increases every year around the world. Traffic accidents can be reduced by detecting driver fatigue. This article describes machine learning for sleep detection. Face detection is used to detect the driver's eye area and use this as a reference for eye tracking in subsequent frames. Finally, visual images are used to detect sleep and a warning system is created. This method is divided into three stages: face detection, eye detection, and fatigue detection. Image processing is used to recognize the driver's face and then extract the image of the driver's eyes to detect fatigue. HAAR face detection algorithm outputs the image and then adjusts face detection based on the output. CHT is then used to track the eyes of the visible face. Check the eyes using EAR (Early Evaluation). The proposed system was tested using the proposed system on a Raspberry pi 3 Model B with 1 GB RAM using Logitech HD Webcam C270. According to some video tests, average eye contact and tracking accuracy can reach 95.0%. Therefore, it is a cheaper and better solution for a tired driver to ask to find the road immediately.

Studying Drowsiness Detection Performance While Driving Through Scalable Machine Learning Models Using Electroencephalography

Driver drowsiness is a significant concern and one of the leading causes of traffic accidents. Advances in cognitive neuroscience and computer science have enabled the detection of drivers’ drowsiness using Brain-Computer Interfaces (BCIs) and Machine Learning (ML). However, the literature lacks a comprehensive evaluation of drowsiness detection performance using a heterogeneous set of ML algorithms, being also necessary to study the performance of scalable ML models suitable for groups of subjects. To address these limitations, this work presents an intelligent framework employing BCIs and features based on electroencephalography for detecting drowsiness in driving scenarios. The SEED-VIG dataset is used to evaluate the best-performing models for individual subjects and groups. Results show that Random Forest (RF) outperformed other models used in the literature, such as Support Vector Machine (SVM), with a 78% f1-score for individual models. Regarding scalable models, RF reached a 79% f1-score, demonstrating the effectiveness of these approaches. This publication highlights the relevance of exploring a diverse set of ML algorithms and scalable approaches suitable for groups of subjects to improve drowsiness detection systems and ultimately reduce the number of accidents caused by driver fatigue. The lessons learned from this study show that not only SVM but also other models not sufficiently explored in the literature are relevant for drowsiness detection. Additionally, scalable approaches are effective in detecting drowsiness, even when new subjects are evaluated. Thus, the proposed framework presents a novel approach for detecting drowsiness in driving scenarios using BCIs and ML.

Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion.

Existing vision-based fatigue detection methods commonly utilize RGB cameras to extract facial and physiological features for monitoring driver fatigue. These features often include single indicators such as eyelid movement, yawning frequency, and heart rate. However, the accuracy of RGB cameras can be affected by factors like varying lighting conditions and motion. To address these challenges, we propose a non-invasive method for multi-modal fusion fatigue detection called RPPMT-CNN-BiLSTM. This method incorporates a feature extraction enhancement module based on the improved Pan-Tompkins algorithm and 1D-MTCNN. This enhances the accuracy of heart rate signal extraction and eyelid features. Furthermore, we use one-dimensional neural networks to construct two models based on heart rate and PERCLOS values, forming a fatigue detection model. To enhance the robustness and accuracy of fatigue detection, the trained model data results are input into the BiLSTM network. This generates a time-fitting relationship between the data extracted from the CNN, allowing for effective dynamic modeling and achieving multi-modal fusion fatigue detection. Numerous experiments validate the effectiveness of the proposed method, achieving an accuracy of 98.2% on the self-made MDAD (Multi-Modal Driver Alertness Dataset). This underscores the feasibility of the algorithm. In comparison with traditional methods, our approach demonstrates higher accuracy and positively contributes to maintaining traffic safety, thereby advancing the field of smart transportation.

A Novel Approach for Automatic Detection of Driver Fatigue Using EEG Signals Based on Graph Convolutional Networks.

Nowadays, the automatic detection of driver fatigue has become one of the important measures to prevent traffic accidents. For this purpose, a lot of research has been conducted in this field in recent years. However, the diagnosis of fatigue in recent research is binary and has no operational capability. This research presents a multi-class driver fatigue detection system based on electroencephalography (EEG) signals using deep learning networks. In the proposed system, a standard driving simulator has been designed, and a database has been collected based on the recording of EEG signals from 20 participants in five different classes of fatigue. In addition to self-report questionnaires, changes in physiological patterns are used to confirm the various stages of weariness in the suggested model. To pre-process and process the signal, a combination of generative adversarial networks (GAN) and graph convolutional networks (GCN) has been used. The proposed deep model includes five convolutional graph layers, one dense layer, and one fully connected layer. The accuracy obtained for the proposed model is 99%, 97%, 96%, and 91%, respectively, for the four different considered practical cases. The proposed model is compared to one developed through recent methods and research and has a promising performance.

SleepyWheels: An ensemble model for drowsiness detection leading to accident prevention

<p>Approximately 30% of traffic accidents in the world are attributed to driver drowsiness. Various approaches have been suggested by different research teams to detect drowsiness, but their methods have drawbacks. Some involve invasive techniques that cause driver discomfort, some involve too many false positives that cause distractions while driving while others rely on complex models that are too resource-intensive. In this paper, we present SleepyWheels, a novel drowsiness detection system. Our key insight is the combined use of two lightweight neural networks: a binary classifier and a facial landmark detector. This innovative approach minimizes false positives and proves resilient across diverse testing scenarios, such as different camera positions, variations in skin tone and when there is obscurement of facial features by objects like eyewear. Research outcomes include a working prototype of the system, a custom dataset for training the classifier and a trained model, which attains an impressive 97% accuracy rate. Deployment and testing were performed on Windows 10 but the system can be deployed on edge devices like Raspberry Pi. The lightweight nature of the models unlocks possibilities of deployment on mobile and embedded devices for use in vehicles.</p>

IoT-Assisted Automatic Driver Drowsiness Detection through Facial Movement Analysis Using Deep Learning and a U-Net-Based Architecture

The main purpose of a detection system is to ascertain the state of an individual’s eyes, whether they are open and alert or closed, and then alert them to their level of fatigue. As a result of this, they will refrain from approaching an accident site. In addition, it would be advantageous for people to be promptly alerted in real time before the occurrence of any calamitous events affecting multiple people. The implementation of Internet-of-Things (IoT) technology in driver action recognition has become imperative due to the ongoing advancements in Artificial Intelligence (AI) and deep learning (DL) within Advanced Driver Assistance Systems (ADAS), which are significantly transforming the driving encounter. This work presents a deep learning model that utilizes a CNN–Long Short-Term Memory network to detect driver sleepiness. We employ different algorithms on datasets such as EM-CNN, VGG-16, GoogLeNet, AlexNet, ResNet50, and CNN-LSTM. The aforementioned algorithms are used for classification, and it is evident that the CNN-LSTM algorithm exhibits superior accuracy compared to alternative deep learning algorithms. The model is provided with video clips of a certain period, and it distinguishes the clip by analyzing the sequence of motions exhibited by the driver in the video. The key objective of this work is to promote road safety by notifying drivers when they exhibit signs of drowsiness, minimizing the probability of accidents caused by fatigue-related disorders. It would help in developing an ADAS that is capable of detecting and addressing driver tiredness proactively. This work intends to limit the potential dangers associated with drowsy driving, hence promoting enhanced road safety and a decrease in accidents caused by fatigue-related variables. This work aims to achieve high efficacy while maintaining a non-intrusive nature. This work endeavors to offer a non-intrusive solution that may be seamlessly integrated into current automobiles, hence enhancing accessibility to a broader spectrum of drivers through the utilization of facial movement analysis employing CNN-LSTM and a U-Net-based architecture.

Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking.

EEG-based unimodal method has demonstrated substantial success in the detection of driving fatigue. Nonetheless, the data from a single modality might be not sufficient to optimize fatigue detection due to incomplete information. To address this limitation and enhance the performance of driving fatigue detection, a novel multimodal architecture combining electroencephalography (EEG) and eye tracking data was proposed in this study. Specifically, EEG and eye tracking data were separately input into encoders, generating two one-dimensional (1D) features. Subsequently, these 1D features were fed into a cross-modal predictive alignment module to improve fusion efficiency and two 1D attention modules to enhance feature representation. Furthermore, the fused features were recognized by a linear classifier. To evaluate the effectiveness of the proposed multimodal method, comprehensive validation tasks were conducted, including intra-session, cross-session, and cross-subject evaluations. In the intra-session task, the proposed architecture achieves an exceptional average accuracy of 99.93%. Moreover, in the cross-session task, our method results in an average accuracy of 88.67%, surpassing the performance of EEG-only approach by 8.52%, eye tracking-only method by 5.92%, multimodal deep canonical correlation analysis (DCCA) technique by 0.42%, and multimodal deep generalized canonical correlation analysis (DGCCA) approach by 0.84%. Similarly, in the cross-subject task, the proposed approach achieves an average accuracy of 78.19%, outperforming EEG-only method by 5.87%, eye tracking-only approach by 4.21%, DCCA method by 0.55%, and DGCCA approach by 0.44%. The experimental results conclusively illustrate the superior effectiveness of the proposed method compared to both single modality approaches and canonical correlation analysis-based multimodal methods. Overall, this study provides a new and effective strategy for driving fatigue detection.