Mental Fatigue Detection Research Articles

Mental fatigue detection using a wearable commodity device and machine learning

Mental fatigue detection using a wearable commodity device and machine learning

Detection of mental fatigue state using heart rate variability and eye metrics during simulated flight

AbstractPilot mental fatigue is a growing concern in the aviation field due to its significant contributions to human errors and aviation accidents. Long work hours, sleep loss, circadian rhythm disruption, and workload are well‐known reasons, but there is a need to accurately detect pilot mental fatigue to improve aviation safety. However, due to the highly restricted cockpit environment and the complex nature of mental fatigue, feasible in‐flight detection remains under‐investigated. The objective of this study is to define a promising approach for mental fatigue detection based on psychophysiological measurements in flying‐relevant environments. Eleven participants engaged in a simulated flight experiment, where several conventional heart rate variability (HRV) and ocular indices were examined to study their relevance to mental fatigue. Additionally, a Toeplitz Inverse Covariance‐Based Clustering (TICC) method was performed to determine the ground truth, after which supervised machine learning was adopted to enable automated mental fatigue detection using HRV and eye metrics. Results showed that HRV and eye metrics were sensitive to the mental fatigue induced by prolonged flight‐relevant tasks. The TICC method helped determine the ground truth for mental fatigue and identify its three distinct levels. Furthermore, a supervised learning‐based detection of mental fatigue was achieved using a support vector machine with the greatest detection accuracy of 91.8%. The findings and methodology of this study provide new insights into the fatigue countermeasures in restricted cockpit environment and lay the groundwork for further explorations into the mental fatigue induced by prolonged flight missions.

Online Mental Fatigue Monitoring via Indirect Brain Dynamics Evaluation

Driver mental fatigue leads to thousands of traffic accidents. The increasing quality and availability of low-cost electroencephalogram (EEG) systems offer possibilities for practical fatigue monitoring. However, non-data-driven methods, designed for practical, complex situations, usually rely on handcrafted data statistics of EEG signals. To reduce human involvement, we introduce a data-driven methodology for online mental fatigue detection: self-weight ordinal regression (SWORE). Reaction time (RT), referring to the length of time people take to react to an emergency, is widely considered an objective behavioral measure for mental fatigue state. Since regression methods are sensitive to extreme RTs, we propose an indirect RT estimation based on preferences to explore the relationship between EEG and RT, which generalizes to any scenario when an objective fatigue indicator is available. In particular, SWORE evaluates the noisy EEG signals from multiple channels in terms of two states: shaking state and steady state. Modeling the shaking state can discriminate the reliable channels from the uninformative ones, while modeling the steady state can suppress the task-nonrelevant fluctuation within each channel. In addition, an online generalized Bayesian moment matching (online GBMM) algorithm is proposed to online-calibrate SWORE efficiently per participant. Experimental results with 40 participants show that SWORE can maximally achieve consistent with RT, demonstrating the feasibility and adaptability of our proposed framework in practical mental fatigue estimation.

Research on Channel Selection and Multi-Feature Fusion of EEG Signals for Mental Fatigue Detection.

With the rapid development of modern social science and technology, the pace of life is getting faster, and brain fatigue has become a sub-health state that seriously affects the normal life of people. Electroencephalogram (EEG) signals reflect changes in the central nervous system. Using EEG signals to assess mental fatigue is a research hotspot in related fields. Most existing fatigue detection methods are time-consuming or don’t achieve satisfactory results due to insufficient features extracted from EEG signals. In this paper, a 2-back task is designed to induce fatigue. The weight value of each channel under a single feature is calculated by ReliefF algorithm. The classification accuracy of each channel under the corresponding features is analyzed. The classification accuracy of each single channel is combined to perform weighted summation to obtain the weight value of each channel. The first half channels sorted in descending order based on the weight value is chosen as the common channels. Multi-features in frequency and time domains are extracted from the common channel data, and the sparse representation method is used to perform feature fusion to obtain sparse fused features. Finally, the SRDA classifier is used to detect the fatigue state. Experimental results show that the proposed methods in our work effectively reduce the number of channels for computation and also improve the mental fatigue detection accuracy.

Predicting task performance from biomarkers of mental fatigue in global brain activity

Objective. Detection and early prediction of mental fatigue (i.e. shifts in vigilance), could be used to adapt neuromodulation strategies to effectively treat patients suffering from brain injury and other indications with prominent chronic mental fatigue. Approach. In this study, we analyzed electrocorticography (ECoG) signals chronically recorded from two healthy non-human primates (NHP) as they performed a sustained attention task over extended periods of time. We employed a set of spectrotemporal and connectivity biomarkers of the ECoG signals to identify periods of mental fatigue and a gradient boosting classifier to predict performance, up to several seconds prior to the behavioral response. Main results. Wavelet entropy and the instantaneous amplitude and frequency were among the best single features across sessions in both NHPs. The classification performance using higher order spectral-temporal (HOST) features was significantly higher than that of conventional spectral power features in both NHPs. Across the 99 sessions analyzed, average F1 scores of 77.5% ± 8.2% and 91.2% ± 3.6%, and accuracy of 79.5% ± 8.9% and 87.6% ± 3.9% for the classifier were obtained for each animal, respectively. Significance. Our results here demonstrate the feasibility of predicting performance and detecting periods of mental fatigue by analyzing ECoG signals, and that this general approach, in principle, could be used for closed-loop control of neuromodulation strategies.

A novel method of mental fatigue detection based on CNN and LSTM

Mental fatigue is a state that may occur due to excessive work or long-term stress. Electroencephalography (EEG) is considered a reliable standard for mental fatigue detection. The existing EEG fatigue detection methods mainly use traditional machine learning models to classify mental fatigue after manual feature extraction. However, manual feature extraction is difficult and complicated. The quality of feature extraction largely determines the quality of the model. In this article, we collected EEG signals from 30 medical staff. The wavelet threshold denoising method was then applied to the measured EEG signal data to denoise the original EEG data, and a method based on a convolution and long short-term memory (CNN + LSTM) neural network to determine the fatigue state of medical staff. The extensive experiment on the established dataset clearly proves the advancement of our proposed algorithm compared to other neural network-based methods. Compared with the existing DNN, CNN and LSTM, the proposed model can quickly learn the information before and after the time series, so as to obtain higher classification accuracy.

A novel method of mental fatigue detection based on CNN and LSTM

Mental fatigue is a state that may occur due to excessive work or long-term stress. Electroencephalography (EEG) is considered a reliable standard for mental fatigue detection. The existing EEG fatigue detection methods mainly use traditional machine learning models to classify mental fatigue after manual feature extraction. However, manual feature extraction is difficult and complicated. The quality of feature extraction largely determines the quality of the model. In this article, we collected EEG signals from 30 medical staff. The wavelet threshold denoising method was then applied to the measured EEG signal data to denoise the original EEG data, and a method based on a convolution and long short-term memory (CNN + LSTM) neural network to determine the fatigue state of medical staff. The extensive experiment on the established dataset clearly proves the advancement of our proposed algorithm compared to other neural network-based methods. Compared with the existing DNN, CNN and LSTM, the proposed model can quickly learn the information before and after the time series, so as to obtain higher classification accuracy.

Audio, Visual, and Electrodermal Arousal Signals as Predictors of Mental Fatigue Following Sustained Cognitive Work.

Lapses in vigilance and slowed reactions due to mental fatigue can increase risk of accidents and injuries and degrade performance. This paper describes a method for rapid, unobtrusive detection of mental fatigue based on changes in electrodermal arousal (EDA), and changes in neuromotor coordination derived from speaking. Twenty-nine Soldiers completed a 2-hour battery of cognitive tasks intended to induce fatigue. Behavioral markers derived from audio and video during speech were acquired before and after the 2hour cognitive load tasks, as was EDA. Exposure to cognitive load produced detectable increases in neuromotor variability in speech and facial measures after load and even after a recovery period. A Gaussian mixture model classifier with crossvalidation and fusion across speech, video, and EDA produced an accuracy of AUC=0.99 in detecting a change in cognitive fatigue relative to a personalized baseline.

Using EEG for Mental Fatigue Assessment: A Comprehensive Look Into the Current State of the Art

This paper provides a brief survey of recent developments on the use of electroencephalogram (EEG) sensors for detecting mental fatigue (MF) in human operators during tasks involving human–machine interaction. This research topic has received much attention since there is a consensus among experts on the increasing relation between human failure and accidents in safety-critical tasks. MF is one of the most influential aspects leading to human failure and the most reliable way to assess it is using operator's physiological data, especially EEG. In the past few decades, hundreds of publications have explored the use of EEG alone or together with other objective and subjective measures for assessing MF, drowsiness, and tiredness in human operators. With recent improvements in data preprocessing, feature extraction, and classification algorithms, the monitoring and mitigation of MF in real time has become a reality. This trend is mainly due to the increasing use of machine learning techniques. This paper provides a comprehensive look at the current state of the art in the field of MF detection using EEG, identifying the currently used technique, algorithms, and methods and possible trends and promising areas for further research. The paper is concluded by suggesting a kernel partial least squares discrete-output linear regression based model as an all-around good option for an MF assessment system.

Proactive mental fatigue detection of traffic control operators using bagged trees and gaze-bin analysis

Most of existing eye movement-based fatigue detectors utilize statistical analysis of fixations, saccades, and blinks as inputs. Nevertheless, these parameters require long recording time and heavily depend on eye trackers. In an effort to facilitate proactive detection of mental fatigue, we introduced a complemental fatigue indicator, named gaze-bin analysis, which simply presents the eye-tracking data with histograms. A method which engaged the gaze-bin analysis as inputs of semisupervised bagged trees was developed. A case study in a vessel traffic service center demonstrated that this approach can alleviate the burden of manual labeling as well as improve the performance of fatigue detection model. In addition, the results show that the approach can achieve an excellent accuracy of 89%, which outperformed other methods. In general, this study provided a complemental indicator for detecting mental fatigue as well as enabled the application of a low sampling rate eye tracker in the traffic control center.

A Review on Features used for EEG-based Mental Fatigue Detection

A Review on Features used for EEG-based Mental Fatigue Detection

Neural Mechanisms of Mental Fatigue Revisited: New Insights from the Brain Connectome

Abstract Maintaining sustained attention during a prolonged cognitive task often comes at a cost: high levels of mental fatigue. Heuristically, mental fatigue refers to a feeling of tiredness or exhaustion, and a disengagement from the task at hand; it manifests as impaired cognitive and behavioral performance. In order to effectively reduce the undesirable yet preventable consequences of mental fatigue in many real-world workspaces, a better understanding of the underlying neural mechanisms is needed, and continuous efforts have been devoted to this topic. In comparison with conventional univariate approaches, which are widely utilized in fatigue studies, convergent evidence has shown that multivariate functional connectivity analysis may lead to richer information about mental fatigue. In fact, mental fatigue is increasingly thought to be related to the deviated reorganization of functional connectivity among brain regions in recent studies. In addition, graph theoretical analysis has shed new light on quantitatively assessing the reorganization of the brain functional networks that are modulated by mental fatigue. This review article begins with a brief introduction to neuroimaging studies on mental fatigue and the brain connectome, followed by a thorough overview of connectome studies on mental fatigue. Although only a limited number of studies have been published thus far, it is believed that the brain connectome can be a useful approach not only for the elucidation of underlying neural mechanisms in the nascent field of neuroergonomics, but also for the automatic detection and classification of mental fatigue in order to address the prevention of fatigue-related human error in the near future.

Study on the Preferred Application-Oriented Index for Mental Fatigue Detection.

Most of the research on mental fatigue evaluation has mainly concentrated on some indexes that require sophisticated and large instruments that make the detection of mental fatigue cumbersome, time-consuming, and difficult to apply on a large scale. A quick and sensitive mental fatigue detection index is necessary so that mentally fatigued workers can be alerted in time and take corresponding countermeasures. However, to date, no studies have compared the sensitivity of common objective evaluation indexes. To solve these problems, this study recruited 56 human subjects. These subjects were evaluated using six fatigue indexes: the Stanford sleepiness scale, digital span, digital decoding, short-term memory, critical flicker fusion frequency (CFF), and speed perception deviation. The results of the fatigue tests before and after mental fatigue were compared, and a one-way analysis of variance (ANOVA) was performed on the speed perception deviation. The results indicated the significance of this index. Considering individual differences, the relative fatigue index (RFI) was proposed to compare the sensitivity of the indexes. The results showed that when the self-rated fatigue grade changed from non-fatigue to mild fatigue, the ranges of RFI values for digital span, digital decoding, short-term memory, and CFF were 0.175–0.258, 0.194–0.316, 0.068–0.139, and 0.055–0.075, respectively. Correspondingly, when the self-rated fatigue grade changed to severe fatigue, the ranges of RFI values for the above indexes were 0.415–0.577, 0.482–0.669, 0.329–0.396, and 0.114–0.218, respectively. These results suggest that the sensitivity of the digital decoding, digital span, short-term memory, and CFF decreased sequentially when the self-evaluated fatigue grade changed from no fatigue to mild or severe fatigue. The RFI individuality of the speed perception deviation is highly variable and is not suitable as an evaluation index. In mental fatigue testing, digital decoding testing can provide faster, more convenient, and more accurate results.

Research on mental fatigue information transmission integration mechanism based on theta-gamma phase amplitude coupling

Mental fatigue is a subjective fatigue state caused by long-term brain activity, which is the core of health problems among brainworkers. However, its influence on the process of brain information transmission integration is not clear. In this paper, phase amplitude coupling (PAC) between theta and gamma rhythm was used to study the electroencephalogram (EEG) data recorded before and after mental fatigue, so as to explain the effect of mental fatigue on brain information transmission mechanism. The experiment used a 4-hour professional English reading to induce brain fatigue. EEG signals of 14 male undergraduate volunteers before and after mental fatigue were recorded by Neuroscan EEG system. Phase amplitude coupling value was calculated and analyzed. t test was used to compare the results between two states. The results showed that theta phase of more than 90% of the electrodes in the whole brain area jointly modulated gamma amplitude of the right central area and the right parietal area, and the coupling effect among different brain regions significantly decreased ( P < 0.05) when participants had felt mental fatigue. This paper shows that phase amplitude coupling can explain the influence of mental fatigue on information transmission mechanism. It could be an important indicator for mental fatigue detection. On the other hand, the results also provide a new measure to evaluate the effect of neuromodulation in relieving mental fatigue.

Detection of mental fatigue state with wearable ECG devices

Overwork-related disorders, such as cerebrovascular/cardiovascular diseases (CCVD) and mental disorders due to overwork, are a major occupational and public health issue worldwide, particularly in East Asian countries. Since wearable smart devices are inexpensive, convenient, popular and widely available today, we were interested in investigating the possibility of using wearable smart electrocardiogram (ECG) devices to detect the mental fatigue state. In total, 35 healthy participants were recruited from a public university in East China. Throughout the entire experiment, each participant wore a wearable device that was further linked to a smartphone to upload the data based on Bluetooth transmission. To manipulate the fatigue state, each participant was asked to finish a quiz, which lasted for approximately 80 min, with 30 logical referential and computing problems and 25 memory tests. Eight heart rate variability (HRV) indicators namely NN.mean (mean of normal to normal interval), rMSSD (root mean square of successive differences), PNN50 (the proportion of NN50 divided by total number of NNs), TP (total spectral power), HF (high frequency from 0.15 Hz to 0.4 Hz), LF (low frequency from 0.04 Hz to 0.15 Hz), VLF (very low frequency from 0.0033 Hz to 0.04 Hz) and the LF/HF ratio were collected at intervals of 5 min throughout the entire experiment. After the feature selection was performed, six indicators remained for further analysis, which were the NN.mean, rMSSD, PNN50, TP, LF, and VLF. Four algorithms, support vector machine (SVM), K-nearest neighbor (KNN), naïve Bayes (NB), and logistic regression (LR), were used to build classifiers that automatically detected the fatigue state. The best performance was achieved by KNN, which had a CV accuracy of 75.5%. The NN.mean, PNN50, TP and LF were the most important HRV indicators for mental fatigue detection. KNN performed the best among the four algorithms and had an average CV accuracy of 65.37% for all of the possible feature combinations.

A new method for automatically modelling brain functional networks

Abstract Traditional methods for constructing brain functional network often need to artificially set a certain threshold, which requires professional and technical personnel to do this work. In order to overcome this deficiency, this study proposed a new method that can automatically construct brain functional network from electroencephalogram (EEG) data, based on positional relations among the vertices and network motif theories. To verify this method, resting state and task state EEG data were converted into brain functional networks with both the new method and traditional methods to explore the discrepancies of network features. The results showed that the mean physical distance increased with the increasing of network edges, evidently suggesting that higher weights of the edges have shorter physical distances, which is the direct model foundation. Besides, consistent results of network features were obtained among these methods, especially in weighted networks, indicating that this new method had the same capacity in accurately characterizing network features compared with the traditional methods. Moreover, this new method can efficiently distinguish the networks that have big differences in the weights, if the network has higher weights, the corresponding network would have more edges, which is in line with one of the traditional methods that using a threshold of weight. We also applied this model in mental fatigue detection, and the results of network characteristics, which obtained from the model and traditional method, have the same variation tendency, approximate values, and similar statistical differences, demonstrating that the proposed model can replace the traditional methods in differentiating similar brain functional states. The new method have potential applications in real-time brain functional networks construction.

Functional Connectivity Analysis of Mental Fatigue Reveals Different Network Topological Alterations Between Driving and Vigilance Tasks.

Despite the apparent importance of mental fatigue detection, a reliable application is hindered due to the incomprehensive understanding of the neural mechanisms of mental fatigue. In this paper, we investigated the topological alterations of functional brain networks in the theta band (4 - 7 Hz) of electroencephalography (EEG) data from 40 male subjects undergoing two distinct fatigue-inducing tasks: a low-intensity one-hour simulated driving and a high-demanding half-hour sustained attention task [psychomotor vigilance task (PVT)]. Behaviorally, subjects demonstrated a robust mental fatigue effect, as reflected by significantly declined performances in cognitive tasks prior and post these two tasks. Furthermore, characteristic path length presented a positive correlation with task duration, which led to a significant increase between the first and the last five minutes of both tasks, indicating a fatigue-related disruption in information processing efficiency. However, significantly increased clustering coefficient was revealed only in the driving task, suggesting distinct network reorganizations between the two fatigue-inducing tasks. Moreover, high accuracy (92% for driving; 97% for PVT) was achieved for fatigue classification with apparently different discriminative functional connectivity features. These findings augment our understanding of the complex nature of fatigue-related neural mechanisms and demonstrate the feasibility of using functional connectivity as neural biomarkers for applicable fatigue monitoring.

A novel real-time driving fatigue detection system based on wireless dry EEG.

Development of techniques for detection of mental fatigue has varied applications in areas where sustaining attention is of critical importance like security and transportation. The objective of this study is to develop a novel real-time driving fatigue detection methodology based on dry Electroencephalographic (EEG) signals. The study has employed two methods in the online detection of mental fatigue: power spectrum density (PSD) and sample entropy (SE). The wavelet packets transform (WPT) method was utilized to obtain the (4-7Hz), (8-12Hz) and (13-30Hz) bands frequency components for calculating corresponding PSD of the selected channels. In order to improve the fatigue detection performance, the system was individually calibrated for each subject in terms of fatigue-sensitive channels selection. Two fatigue-related indexes: ( )/ and / were computed and then fused into an integrated metric to predict the degree of driving fatigue. In the case of SE extraction, the mean of SE averaged across two EEG channels ('O1h' and 'O2h') was used for fatigue detection. Ten healthy subjects participated in our study and each of them performed two sessions of simulated driving. In each session, subjects were required to drive simulated car for 90min without any break. The results demonstrate that our proposed methods are effective for fatigue detection. The prediction of fatigue is consistent with the observation of reaction time that was recorded during simulated driving, which is considered as an objective behavioral measure.

A New Method for Human Mental Fatigue Detection with Several EEG Channels

Previous researchers have applied EEG in human mental fatigue detection and concluded that the ratio of slow wave to fast wave EEG activities increased with increasing mental fatigue. But these methods failed to reflect functional interactions between different brain areas. The objective of this paper is to develop an indicator for detecting human mental fatigue with several electroencephalogram (EEG) channels considering functional interactions among the brain. To this end, we collected eighteen participants’ EEG data, and constructed the adjacency matrix on the basis of EEG electrode positions using mutual information (MI) to determine the functional connectivity between all pairwise EEG channels. The maximum eigenvalue of adjacency matrix was discussed during the mental fatigue process and considered as the reflections of the corresponding network features. The results indicated that significant statistical differences of the maximum eigenvalue were found only in alpha1 (8–10 Hz) band during task state. With increasing levels of mental fatigue, the maximum eigenvalue increased, indicating that the maximum eigenvalue can be used as an indicator for mental fatigue estimation taking brain function changes into account. For real-time engineering applications, we reduced the EEG channels by finding the central vertices in the brain functional network using weighted degree centrality. Finally, the channels of F3, F4, C3, C4, P3, P4, Fz, Cz, and Pz were obtained for further applications. We also provided an algorithm for automatic human mental fatigue evaluation with these 9 EEG channels. Our results have potential applications in several domains, including in flight, traffic and industrial human safety.

Classification of EEG based-mental fatigue using principal component analysis and Bayesian neural network.

This paper presents an electroencephalography (EEG) based-classification of between pre- and post-mental load tasks for mental fatigue detection from 65 healthy participants. During the data collection, eye closed and eye open tasks were collected before and after conducting the mental load tasks. For the computational intelligence, the system uses the combination of principal component analysis (PCA) as the dimension reduction method of the original 26 channels of EEG data, power spectral density (PSD) as feature extractor and Bayesian neural network (BNN) as classifier. After applying the PCA, the dimension of the data is reduced from 26 EEG channels in 6 principal components (PCs) with above 90% of information retained. Based on this reduced dimension of 6 PCs of data, during eyes open, the classification pre-task (alert) vs. post-task (fatigue) using Bayesian neural network resulted in sensitivity of 76.8 %, specificity of 75.1% and accuracy of 76% Also based on data from the 6 PCs, during eye closed, the classification between pre- and post-task resulted in a sensitivity of 76.1%, specificity of 74.5% and accuracy of 75.3%. Further, the classification results of using only 6 PCs data are comparable to the result using the original 26 EEG channels. This finding will help in reducing the computational complexity of data analysis based on 26 channels of EEG for mental fatigue detection.