Driver Fatigue State Research Articles

Clustering for mitigating subject variability in driving fatigue classification using electroencephalography source-space functional connectivity features.

While Electroencephalography (EEG)-based driver fatigue state classification models have demonstrated effectiveness, their real-world application remains uncertain. The substantial variability in EEG signals among individuals poses a challenge in developing a universal model, often necessitating retraining with the introduction of new subjects. However, obtaining sufficient data for retraining, especially fatigue data for new subjects, is impractical in real-world settings. In response to these challenges, this paper introduces a hybrid solution for fatigue detection that combines clustering with classification. Unsupervised clustering groups subjects based on their EEG functional connectivity in an alert state, and classification models are subsequently applied to each cluster for predicting alert and fatigue states. Results indicate that classification on clusters achieves higher accuracy than scenarios without clustering, suggesting successful grouping of subjects with similar functional connectivity characteristics through clustering, thereby enhancing the classification process. Furthermore, the proposed hybrid method ensures a practical and realistic retraining process, improving the adaptability and effectiveness of the fatigue detection system in real-world applications.&#xD.

Efficient detection of driver fatigue state based on all-weather illumination scenarios

Among the causes of the annually traffic accidents, driving fatigue is the main culprit. In consequence, it is of great practical significance to carry out the research of driving fatigue detection and early warning system. However, there are still two problems in the latest methods of driving fatigue detection: one is that a single information cannot precisely reflect the actual state of the driver in different fatigue phases, another one is the detection effect is not very well or even difficult to detect under abnormal illumination. In this paper, the multi-task cascaded convolutional networks (MTCNN) and infrared-based remote photo-plethysmography (rPPG) theory are used to extract the driver’s facial and physiological information, and the multi-modal specific fatigue information is deeply excavated, and the multi-modal feature fusion model is constructed to comprehensively analyze the driver’s fatigue variation tendency. Aiming at the matter of low detection accuracy under abnormal illumination, the multi-modal features extracted from visible light images and infrared images are fused by multi-loss reconstruction (MLR) module, and the driving fatigue detection module is established which is based on Bi-LSTM model by utilizing fatigue timing. The experiments were validated under all-weather illumination scenarios and were carried out on the datasets NTHU-DDD, UTA-RLDDD and FAHD. The results show that the multi-modal driving fatigue detection model has better performance than the single-modal model, and the accuracy is improved by 8.1%. In the abnormal illumination such as strong and weak light, the accuracy of the method can reach 91.7% at the highest and 83.6% at the lowest. Meanwhile, in the normal illumination, it can reach 93.2%.

Driver fatigue detection method based on facial features using deep learning

Fatigue driving can result in delayed driver reactions and lack of concentration, increasing the risk of traffic accidents. Therefore, finding effective fatigue detection methods is crucial for accident prevention and ensuring public safety. Traditional driver fatigue detection methods based on facial features are limited by their reliance on manually designed features and classifiers, leading to low accuracy, slow detection speed, and inefficient performance. Hence, methods based on deep learning have gradually become a trend in fatigue-driving detection research. This paper focuses on driver fatigue detection methods based on facial features using deep learning. In this study, we analyze numerous research outcomes on how to accurately extract, analyze, and integrate various facial features, such as eye features, mouth features, facial expressions, and head postures, to precisely determine the driver's fatigue state. Then, we compare the latest deep learning-based technologies. This paper provides an information-rich repository to help researchers select more promising designs, that can drive the widespread development of fatigue-driving detection technologies.

EEG-based detection of driving fatigue using a novel electrode

In view of the serious harm caused by driving fatigue, this paper investigated driving fatigue detection based on electroencephalogram (EEG) features, designed a novel semi-dry electrode for acquisitioning drivers' EEG signals, and performed refined composite multiscale fluctuation dispersion entropy (RCMFDE) feature extraction of the θ and β bands rhythm signals in the acquisitioned EEG signals. The research results showed that the novel semi-dry electrode designed in this paper had the advantages of convenient replenishment of conductive liquid, early warning of insufficient conductive liquid, comfortable to wear compared with the traditional wet and dry electrodes under the premise of being able to collect EEG signals with qualified quality. And compared with traditional methods such as multiscale sample entropy (MSE), multiscale permutation entropy (MPE), multiscale symbolic dynamic entropy (MSDE), multiscale dispersion entropy (MDE), and refined composite multiscale dispersion entropy (RCMDE), the EEG features extracted by the RCMFDE method in this paper have a more obvious fatigue feature tendency, and thus can identify the driving fatigue state more effectively.

Real driving environment EEG-based detection of driving fatigue using the wavelet scattering network

BackgroundDriving fatigue is one of the main factors leading to traffic accidents. So, it is necessary to detect driver fatigue accurately and quickly. New methodTo precisely detect driving fatigue in a real driving environment, this paper adopts a classification method for driving fatigue based on the wavelet scattering network (WSN). Firstly, electroencephalogram (EEG) signals of 12 subjects in the real driving environment are collected and categorized into two states: fatigue and awake. Secondly, the WSN algorithm extracts wavelet scattering coefficients of EEG signals, and these coefficients are used as input in support vector machine (SVM) as feature vectors for classification. ResultsThe results showed that the average classification accuracy of 12 subjects reached 99.33%; the average precision rate reached 99.28%; the average recall rate reached 98.27%; the average F1 score reached 98.74%; and the average classification accuracy of the public data set SEED-VIG reached 99.39%. The average precision, recall rate and F1 score reached 99.27%, 98.41% and 98.83% respectively. Comparison with existing methodsIn addition, the WSN algorithm is compared with traditional convolutional neural network (CNN), Sparse-deep belief networks (SDBN), Spatio-temporal convolutional neural networks (STCNN), Long short-term memory (LSTM), and other methods, and it is found that WSN has higher classification accuracy. ConclusionFurthermore, this method has good versatility, providing excellent recognition effect on small sample data sets, and fast running time, making it convenient for real-time online monitoring of driver fatigue. Therefore, the WSN algorithm is promising in efficiently detecting driving fatigue state of drivers in real environments, contributing to improved traffic safety.

A Feature Fusion Method for Driving Fatigue of Shield Machine Drivers Based on Multiple Physiological Signals and Auto-Encoder

The driving fatigue state of shield machine drivers directly affects the safe operation and tunneling efficiency of shield machines during metro construction. To cope with the problem that it is challenging to simulate the working conditions and operation process of shield machine drivers using driving simulation platforms and that the existing fatigue feature fusion methods usually show low recognition accuracy, shield machine drivers at Shenyang metro line 4 in China were taken as the research subjects, and a multi-modal physiological feature fusion method based on an L2-regularized stacked auto-encoder was designed. First, the ErgoLAB cloud platform was used to extract the combined energy feature (E), the reaction time, the HRV (heart rate variability) time-domain SDNN (standard deviation of normal-to-normal intervals) index, the HRV frequency-domain LF/HF (energy ratio of low frequency to high frequency) index and the pupil diameter index from EEG (electroencephalogram) signals, skin signals, pulse signals and eye movement data, respectively. Second, the physiological signal characteristics were extracted based on the WPT (wavelet packet transform) method and time–frequency analysis. Then, a method for driving fatigue feature fusion based on an auto-encoder was designed aiming at the characteristics of the L2-regularization method to solve the over-fitting problem of small sample data sets in the process of model training. The optimal hyper-parameters of the model were verified with the experimental method of the control variable, which reduces the loss of multi-modal feature data in compression fusion and the information loss rate of the fused index. The results show that the method proposed outperforms its competitors in recognition accuracy and can effectively reduce the loss rate of deep features in existing decision-making-level fusion.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks.

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

Fatigue driving detection method based on Time-Space-Frequency features of multimodal signals

Fatigue detection for drivers in public transportation is crucial. To effectively detect the driver's fatigue state, we investigated the deep learning-based fatigue detection method and proposed a multimodality signal fatigue detection method. In the proposed method, the convolutional autoencoder (CAE) is used to fuse electroencephalogram (EEG) and electrooculography (EOG) signal features, and the convolutional neural network (CNN) is used to maintain spatial locality. After that, the fusion features are input into the recurrent neural network (RNN) for fatigue recognition. We tested the proposed algorithmic framework on the SEED-VIG dataset and evaluated it using two statistical indicators, root mean square error (RMSE) and correlation coefficient (COR), achieving the mean RMSE/COR of 0.10/0.93 and 0.11/0.88 on single modality EOG and EEG features, respectively, and improved performance to 0.08/0.96 on multimodality features. In addition, this paper analyzes the effect of different signal features on recognition results, and the comparison illustrates that the performance of the model using multimodality features is better than that of signle modality features. The experimental results show that the algorithm framework proposed in this paper outperforms other recognition algorithms, which also proves the effectiveness of the algorithm applied to fatigue driving detection.

Driver Emotion and Fatigue State Detection Based on Time Series Fusion

Studies have shown that driver fatigue or unpleasant emotions significantly increase driving risks. Detecting driver emotions and fatigue states and providing timely warnings can effectively minimize the incidence of traffic accidents. However, existing models rarely combine driver emotion and fatigue detection, and there is space to improve the accuracy of recognition. In this paper, we propose a non-invasive and efficient detection method for driver fatigue and emotional state, which is the first time to combine them in the detection of driver state. Firstly, the captured video image sequences are preprocessed, and Dlib (image open source processing library) is used to locate face regions and mark key points; secondly, facial features are extracted, and fatigue indicators, such as driver eye closure time (PERCLOS) and yawn frequency are calculated using the dual-threshold method and fused by mathematical methods; thirdly, an improved lightweight RM-Xception convolutional neural network is introduced to identify the driver’s emotional state; finally, the two indicators are fused based on time series to obtain a comprehensive score for evaluating the driver’s state. The results show that the fatigue detection algorithm proposed in this paper has high accuracy, and the accuracy of the emotion recognition network reaches an accuracy rate of 73.32% on the Fer2013 dataset. The composite score calculated based on time series fusion can comprehensively and accurately reflect the driver state in different environments and make a contribution to future research in the field of assisted safe driving.

A real-time driver fatigue identification method based on GA-GRNN.

It is of great practical and theoretical significance to identify driver fatigue state in real time and accurately and provide active safety warning in time. In this paper, a non-invasive and low-cost method of fatigue driving state identification based on genetic algorithm optimization of generalized regression neural network model is proposed. The specific work is as follows: (1) design simulated driving experiment and real driving experiment, determine the fatigue state of drivers according to the binary Karolinska Sleepiness Scale (KSS), and establish the fatigue driving sample database. (2) Improved Multi-Task Cascaded Convolutional Networks (MTCNN) and applied to face detection. Dlib library was used to extract the coordinate values of face feature points, collect the characteristic parameters of driver's eyes and mouth, and calculate the Euler Angle parameters of head posture. A fatigue identification model was constructed by using multiple characteristic parameters. (3) Genetic Algorithm (GA) was used to find the optimal smooth factor of Generalized Regression Neural Network (GRNN) and construct GA-GRNN fatigue driving identification model. Compared with K-Nearest Neighbor (KNN), Random Forest (RF), and GRNN fatigue driving identification algorithms. GA-GRNN has the best generalization ability and high stability, with an accuracy of 93.3%. This study provides theoretical and technical support for the application of driver fatigue identification.

Directed Brain Network Analysis for Fatigue Driving Based on EEG Source Signals.

Fatigue driving is one of the major factors that leads to traffic accidents. Long-term monotonous driving can easily cause a decrease in the driver’s attention and vigilance, manifesting a fatigue effect. This paper proposes a means of revealing the effects of driving fatigue on the brain’s information processing abilities, from the aspect of a directed brain network based on electroencephalogram (EEG) source signals. Based on current source density (CSD) data derived from EEG signals using source analysis, a directed brain network for fatigue driving was constructed by using a directed transfer function. As driving time increased, the average clustering coefficient as well as the average path length gradually increased; meanwhile, global efficiency gradually decreased for most rhythms, suggesting that deep driving fatigue enhances the brain’s local information integration abilities while weakening its global abilities. Furthermore, causal flow analysis showed electrodes with significant differences between the awake state and the driving fatigue state, which were mainly distributed in several areas of the anterior and posterior regions, especially under the theta rhythm. It was also found that the ability of the anterior regions to receive information from the posterior regions became significantly worse in the driving fatigue state. These findings may provide a theoretical basis for revealing the underlying neural mechanisms of driving fatigue.

Research on driving fatigue detection based on basic scale entropy and MVAR-PSI

In long-term continuous driving, driving fatigue is the main cause of traffic accidents. Therefore, accurate and rapid detection of driver mental fatigue is of great significance to traffic safety. In our study, the electroencephalogram (EEG) signals of subjects were preprocessed to remove interference signals. The Butterworth band-pass filter is used to extract the EEG signals of α and β rhythms, and then the basic scale entropy of α and β rhythms is used as driving fatigue characteristics. In addition, combined with the fast multiple autoregressive (MVAR) model and phase slope index (PSI), short-term data is used to accurately estimate the effective connectivity of EEG signals between different channels, and analyzed the causality flow direction in the left and right prefrontal regions of drivers at different driving stages. Further comprehensive analysis of the driver’s driving fatigue state in the continuous driving phase. Finally, the correlation coefficient value between the parameter pairs (basic scale entropy, clustering coefficient, global efficiency) is calculated. The results showed that the causality flow outflow degree of prefrontal lobe decreased during the transition from sober driving state to tired driving state. The left and right prefrontal lobes were the source of causality in sober driving state, and gradually became the target of causality with the occurrence of driving fatigue. The results showed that when transitioning from a waking state to a fatigued driving state, the causal flow direction out-degree value of the prefrontal cortex on a declining curve, and the left and right prefrontal cortex exhibited the causal source in the awake driving state, which gradually changed into the causal target along with the occurrence of driving fatigue. The three parameters of basic scale entropy, clustering coefficient and global efficiency are used as driving fatigue characteristics, and every two parameters have strong correlation. It shows that the combination of basic scale entropy and MVAR-PSI method can effectively detect the driver’s long-term driving fatigue state in continuous driving mode.

Driving Detection Based on the Multifeature Fusion

In order to solve the problems of facial feature localization and driver fatigue state identification methods in driving fatigue detection, a driving detection method based on the multifeature fusion was proposed. This method uses a supervised descent algorithm to simultaneously locate multiple facial features of drivers. On the basis of blink, yawn and nod judgment, multiple characteristic values of blink frequency, yawn frequency, and nod frequency of drivers were extracted to establish a fatigue detection sample database, and a naive Bayes classifier was constructed to judge fatigue. When the driver appears fatigue driving, warning information is given in time in order to prevent traffic accidents. The experimental results show that two sample videos were selected for testing. The accuracy rate of video sample 1 and video sample 2 was 94.74% and 95.00%, respectively. Conclusion. In the actual driving environment video test results, the discriminant average accuracy of a driver fatigue state reaches 94.87%, which has a good performance.

Driver Fatigue Detection Method Based on Human Pose Information Entropy

Driver fatigue detection (DFD) is an effective method to prevent traffic accidents. The existing research on DFD using facial features is an effective and noninvasive fatigue detection method. However, this approach is affected by facial occlusions (glasses, sunglasses, masks, etc.) and the large facial pose deformations in the extraction of effective fatigue features. In this paper, we introduce a novel DFD method using human pose information entropy. The method first estimates human pose from video sequences and then uses them as clues to extract multiple fatigue-related features which can reduce the influence of facial occlusion and head pose deformation. Information entropy and sliding window algorithm are applied to analyse and calculate sufficient consecutive video frames to obtain more robust and accurate fatigue-related values than by using a single frame. These information entropy values are combined resorting to the support vector machine (SVM) to recognize the driver fatigue state. Experimental results show that the method can achieve much higher accuracy and robustness, and the detection speed meets the requirements of real time.

Driver drowsiness detection system using machine learning

This feature is used for the safety of the peoples from the accidents. This work shows the development of ADAS ( advance driving assistance system) this focus on the driver drowsiness detection is main aim to alert the driver from the drowsiness state to avoid road accidents. In this we use the Machine Learning to predict the condition and emotions of the driver that will improve the safety on the roads. We use the CNN for this project that will effectively detect the driver fatigue status using driver images. In this project we can use the Electrocardiogram (ECG) for the psychological system to detect drowsiness. Artificial Intelligence means system can automatically learns as well as improve without being programmed. We also use the various machine learning techniques like OpenCv, Keras, tensorflow

Reduction of Artifacts in Capacitive Electrocardiogram Signals of Driving Subjects.

The development of smart cars with e-health services allows monitoring of the health condition of the driver. Driver comfort is preserved by the use of capacitive electrodes, but the recorded signal is characterized by large artifacts. This paper proposes a method for reducing artifacts from the ECG signal recorded by capacitive electrodes (cECG) in moving subjects. Two dominant artifact types are coarse and slow-changing artifacts. Slow-changing artifacts removal by classical filtering is not feasible as the spectral bands of artifacts and cECG overlap, mostly in the band from 0.5 to 15 Hz. We developed a method for artifact removal, based on estimating the fluctuation around linear trend, for both artifact types, including a condition for determining the presence of coarse artifacts. The method was validated on cECG recorded while driving, with the artifacts predominantly due to the movements, as well as on cECG recorded while lying, where the movements were performed according to a predefined protocol. The proposed method eliminates 96% to 100% of the coarse artifacts, while the slow-changing artifacts are completely reduced for the recorded cECG signals larger than 0.3 V. The obtained results are in accordance with the opinion of medical experts. The method is intended for reliable extraction of cardiovascular parameters to monitor driver fatigue status.

Real-time detection method of driver fatigue state based on deep learning of face video

The use of face video information for driver fatigue detection has received extensive attention because of its low cost and non-invasiveness. However, the current vehicle-mounted embedded device has insufficient memory and limited computing power, which cannot complete the real-time detection of driver fatigue based on deep learning. Therefore, this paper designs a lightweight neural network model to solve this problem. The model includes object detection and fatigue detection. First, a lightweight object detection network is designed, which can quickly identify the opening and closing states of the driver’s eyes and mouth in the time series video. Secondly, the EYE-MOUTH (EM) driver fatigue detection model is designed, which encodes the driver’s eye and mouth opening and closing state, and calculates the driver’s PERCLOS (Percentage of Eyelid Closure over the Pupil) and FOM (Frequency of Open Mouth) according to the coding sequence. Finally, the multi-feature fusion judgment algorithm is used to realize the judgment of the driver’s fatigue state. The experimental results show that our method has an accuracy rate of 98.30% for drowsiness and yawning behaviors in a real vehicle environment, and a detection speed of 27FPS, which is better than other advanced methods and meets the requirements of real-time detection.

Optimization of Fatigue Detection Method under Altitude Changes in Plateau Region Based on MTCNN

Fatigue driving is the main cause of traffic accidents, and research on fatigue driving detection algorithms is of great significance to improve road safety. This paper proposes an image processing method based on MTCNN model detection optimization, Perform median filter denoising before P-Net training to improve the detection rate of night faces, then, the ASM algorithm is used to detect the facial feature points, and finally the PERCLOS principle is used to analyze the driving fatigue state. The experimental results show that the method has a high detection rate, can be applied to fatigue detection at different altitudes, and has strong practicability.

Research on Driver Fatigue State Detection Method Based on Deep Learning

Fatigue driving detection is essential to ensure the safety of society and drivers. At present, most fatigue detection methods are relatively traditional and single, and have complex algorithms, low accuracy, and low fault tolerance. Based on the improved Multi-task Cascaded Convolutional Network (MTCNN) to achieve precise positioning of facial feature points, combined with the Res-SE-net model to achieve eye, mouth area and state classification. The model is trained, and finally the driver fatigue is judged based on the PERCLOS rule combined with the OMR rule of mouth opening and closing frequency. Experimental results show that this method can effectively extract fatigue features, has high detection accuracy, meets real-time requirements, and has high robustness to complex environments.

A method for fatigue detection based on Driver's steering wheel grip

We propose a simple method to measure a driver's fatigue state by detecting the driver's grip force on the steering wheel while driving. We tested the grip force of 36 drivers on the steering wheel in conscious states (Alert) and fatigue states under actual road driving conditions. Using the Stanford sleepiness scale (SSS), we divided drivers into Alert Group A, fatigue Group A, and fatigue Group B. During 20-min real-road driving trials, we measured the steering wheel grip force, electroencephalogram index (R = (α + θ)/β), and blink frequency of each driver synchronously. We found that ΔF, the difference between the maximum/minimum grip force and the standard deviation of the grip force, σF, for each driver, strongly correlated with the driver's fatigue state. In the fatigue state, both ΔF and σF increased significantly. We examined these force indices using analysis of variance (ANOVA) and validated them against the R-value, blink frequency, and the driver's self-reported fatigue state. Using the grip force in fatigue detection, our method can achieve an overall recognition rate of 86.6% and an individual recognition rate of 88.3%. These results indicate that this method can effectively detect a driver's fatigue state during actual road driving. This new method has several advantages, such as a high signal-to-noise ratio, simple data collection, and no influence on daily driving. Thus, our proposed method may provide a theoretical foundation for the development of fatigue-detecting steering wheels