Mental Workload Classification Research Articles

ECG and EEG based machine learning models for the classification of mental workload and stress levels for women in different menstrual phases, men, and mixed sexes

ECG and EEG based machine learning models for the classification of mental workload and stress levels for women in different menstrual phases, men, and mixed sexes

EEG-Based Mental Workload Classification Method Based on Hybrid Deep Learning Model Under IoT.

Automatically detecting human mental workload to prevent mental diseases is highly important. With the development of information technology, remote detection of mental workload is expected. The development of artificial intelligence and Internet of Things technology will also enable the identification of mental workload remotely based on human physiological signals. In this article, a method based on the spatial and time-frequency domains of electroencephalography (EEG) signals is proposed to improve the classification accuracy of mental workload. Moreover, a hybrid deep learning model is presented. First, the spatial domain features of different brain regions are proposed. Simultaneously, EEG time-frequency domain information is obtained based on wavelet transform. The spatial and time-frequency domain features are input into two types of deep learning models for mental workload classification. To validate the performance of the proposed method, the Simultaneous Task EEG Workload public database is used. Compared with the existing methods, the proposed approach shows higher classification accuracy. It provides a novel means of assessing mental workload.

Classification of mental workload using brain connectivity and machine learning on electroencephalogram data

Mental workload refers to the cognitive effort required to perform tasks, and it is an important factor in various fields, including system design, clinical medicine, and industrial applications. In this paper, we propose innovative methods to assess mental workload from EEG data that use effective brain connectivity for the purpose of extracting features, a hierarchical feature selection algorithm to select the most significant features, and finally machine learning models. We have used the Simultaneous Task EEG Workload (STEW) dataset, an open-access collection of raw EEG data from 48 subjects. We extracted brain-effective connectivities by the direct directed transfer function and then selected the top 30 connectivities for each standard frequency band. Then we applied three feature selection algorithms (forward feature selection, Relief-F, and minimum-redundancy-maximum-relevance) on the top 150 features from all frequencies. Finally, we applied sevenfold cross-validation on four machine learning models (support vector machine (SVM), linear discriminant analysis, random forest, and decision tree). The results revealed that SVM as the machine learning model and forward feature selection as the feature selection method work better than others and could classify the mental workload levels with accuracy equal to 89.53% (± 1.36).

Assessment of Mental Workload Level Based on PPG Signal Fusion Continuous Wavelet Transform and Cardiopulmonary Coupling Technology

Mental workload is an important predisposing factor for mental illnesses such as depression and is closely related to individual mental health. However, the suboptimal accuracy of utilizing photoplethysmography (PPG) exclusively for mental workload classification has constrained its application within pertinent professional domains. To this end, this paper proposes a signal processing method that combines continuous wavelet transform (CWT) and cardiopulmonary coupling mapping (CPC) to classify mental load via a convolutional neural network (ResAttNet). The method reflects changes in mental workload, as assessed by changes in the association between heart rate variability and respiration. In this paper, the strengths and weaknesses of this method are compared with other traditional psychological workload monitoring methods, such as heart rate variability (HRV), and its validation is performed on the publicly available dataset MAUS. The experiments show that the method is significantly better than previous machine learning methods based on heart rate variability correlation. Meanwhile, the accuracy of the method proposed in this paper reaches 80.5%, which is 6.2% higher than in previous studies. It is comparable to the result of 82.4% for the ECG-based mental workload monitoring system. Therefore, the method of combining CWT and CPC has considerable potential and provides new ideas for mental workload classification.

Mental Workload Assessment Using Machine Learning Techniques Based on EEG and Eye Tracking Data

The main contribution of this study was the concurrent application of EEG and eye tracking techniques during n-back tasks as part of the methodology for addressing the problem of mental workload classification through machine learning algorithms. The experiments involved 15 university students, consisting of 7 women and 8 men. Throughout the experiments, the researchers utilized the n-back memory task and the NASA-Task Load Index (TLX) subjective rating scale to assess various levels of mental workload. The results indicating the relationship between EEG and eye tracking measures and mental workload are consistent with previous research. Regarding the four-class classification task, mental workload level could be predicted with 76.59% accuracy using 34 selected features. This study makes a significant contribution to the literature by presenting a four-class mental workload estimation model that utilizes different machine learning algorithms.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study.

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

Mental workload classification using convolutional neural networks based on fNIRS-derived prefrontal activity

BackgroundFunctional near-infrared spectroscopy (fNIRS) is a tool to assess brain activity during cognitive testing. Despite its usefulness, its feasibility in assessing mental workload remains unclear. This study was to investigate the potential use of convolutional neural networks (CNNs) based on functional near-infrared spectroscopy (fNIRS)-derived signals to classify mental workload in individuals with mild cognitive impairment.MethodsSpatial images by constructing a statistical activation map from the prefrontal activity of 120 subjects with MCI performing three difficulty levels of the N-back task (0, 1, and 2-back) were used for CNNs. The CNNs were evaluated using a 5 and 10-fold cross-validation method.ResultsAs the difficulty level of the N-back task increased, the accuracy decreased and prefrontal activity increased. In addition, there was a significant difference in the accuracy and prefrontal activity across the three levels (p’s < 0.05). The accuracy of the CNNs based on fNIRS-derived spatial images evaluated by 5 and 10-fold cross-validation in classifying the difficulty levels ranged from 0.83 to 0.96.ConclusionfNIRS could also be a promising tool for measuring mental workload in older adults with MCI despite their cognitive decline. In addition, this study demonstrated the feasibility of the classification performance of the CNNs based on fNIRS-derived signals from the prefrontal cortex.

Evidence for modulation of EEG microstates by mental workload levels and task types.

Electroencephalography (EEG) microstate analysis has become a popular tool for studying the spatial and temporal dynamics of large-scale electrophysiological activities in the brain in recent years. Four canonical topographies of the electric field (classes A, B, C, and D) have been widely identified, and changes in microstate parameters are associated with several psychiatric disorders and cognitive functions. Recent studies have reported the modulation of EEG microstate by mental workload (MWL). However, the common practice of evaluating MWL is in a specific task. Whether the modulation of microstate by MWL is consistent across different types of tasks is still not clear. Here, we studied the topographies and dynamics of microstate in two independent MWL tasks: NBack and the multi-attribute task battery (MATB) and showed that the modulation of MWL on microstate topographies and parameters depended on tasks. We found that the parameters of microstates A and C, and the topographies of microstates A, B, and D were significantly different between the two tasks. Meanwhile, all four microstate topographies and parameters of microstates A and C were different during the NBack task, but no significant difference was found during the MATB task. Furthermore, we employed a support vector machine recursive feature elimination procedure to investigate whether microstate parameters were suitable for MWL classification. An averaged classification accuracy of 87% for within-task and 78% for cross-task MWL discrimination was achieved with at least 10 features. Collectively, our findings suggest that topographies and parameters of microstates can provide valuable information about neural activity patterns with a dynamic temporal structure at different levels of MWL, but the modulation of MWL depends on tasks and their corresponding functional systems. Moreover, as a potential indicator, microstate parameters could be used to distinguish MWL.

Enhancing EEG-based cross-day mental workload classification using periodic component of power spectrum

Objective. The day-to-day variability of electroencephalogram (EEG) poses a significant challenge to decode human brain activity in EEG-based passive brain-computer interfaces (pBCIs). Conventionally, a time-consuming calibration process is required to collect data from users on a new day to ensure the performance of the machine learning-based decoding model, which hinders the application of pBCIs to monitor mental workload (MWL) states in real-world settings. Approach. This study investigated the day-to-day stability of the raw power spectral density (PSD) and their periodic and aperiodic components decomposed by the Fitting Oscillations and One-Over-F algorithm. In addition, we validated the feasibility of using periodic components to improve cross-day MWL classification performance. Main results. Compared to the raw PSD (69.9% ± 18.5%) and the aperiodic component (69.4% ± 19.2%), the periodic component had better day-to-day stability and significantly higher cross-day classification accuracy (84.2% ± 11.0%). Significance. These findings indicate that periodic components of EEG have the potential to be applied in decoding brain states for more robust pBCIs.

Assessment of mental workload across cognitive tasks using a passive brain-computer interface based on mean negative theta-band amplitudes.

Brain-computer interfaces (BCI) can provide real-time and continuous assessments of mental workload in different scenarios, which can subsequently be used to optimize human-computer interaction. However, assessment of mental workload is complicated by the task-dependent nature of the underlying neural signals. Thus, classifiers trained on data from one task do not generalize well to other tasks. Previous attempts at classifying mental workload across different cognitive tasks have therefore only been partially successful. Here we introduce a novel algorithm to extract frontal theta oscillations from electroencephalographic (EEG) recordings of brain activity and show that it can be used to detect mental workload across different cognitive tasks. We use a published data set that investigated subject dependent task transfer, based on Filter Bank Common Spatial Patterns. After testing, our approach enables a binary classification of mental workload with performances of 92.00 and 92.35%, respectively for either low or high workload vs. an initial no workload condition, with significantly better results than those of the previous approach. It, nevertheless, does not perform beyond chance level when comparing high vs. low workload conditions. Also, when an independent component analysis was done first with the data (and before any additional preprocessing procedure), even though we achieved more stable classification results above chance level across all tasks, it did not perform better than the previous approach. These mixed results illustrate that while the proposed algorithm cannot replace previous general-purpose classification methods, it may outperform state-of-the-art algorithms in specific (workload) comparisons.

Efficacy of transformer networks for classification of EEG data

With the unprecedented success of transformer networks in natural language processing (NLP), recently, they have been successfully adapted to areas like computer vision, generative adversarial networks (GAN), and reinforcement learning. Classifying electroencephalogram (EEG) data has been challenging and researchers have been overly dependent on pre-processing and hand-crafted feature extraction. Despite having achieved automated feature extraction in several other domains, deep learning has not yet been accomplished for EEG. In this paper, the efficacy of the transformer network for the classification of raw EEG data (cleaned and pre-processed) is explored. The performance of transformer networks was evaluated on a local (age and gender data) and a public dataset (STEW). First, a classifier using a transformer network is built to classify the age and gender of a person with raw resting-state EEG data. Second, the classifier is tuned for mental workload classification with open access raw multi-tasking mental workload EEG data (STEW). The network achieves an accuracy comparable to state-of-the-art accuracy on both the local (Age and Gender dataset; 94.53% (gender) and 87.79% (age)) and the public (STEW dataset; 95.28% (two workload levels) and 88.72% (three workload levels)) dataset. The accuracy values have been achieved using raw EEG data without feature extraction. Results indicate that the transformer-based deep learning models can successfully abate the need for heavy feature-extraction of EEG data for successful classification.

Classification of high mental workload and emotional statuses via machine learning feature extractions in gait

Gait features could help to identify workers' mental workload and different emotional statuses. In the current study, thirty participants were recruited to walk after four emotional states were elicited and the gait data was collected using the optical motion capture system. The 111 posture and kinematic features were extracted from 2 min gait performances. The top 40 ranked features were selected for the Support Vector Machine (SVM) classifier to classify each emotional state. For individual emotional classification, the results revealed that the accuracy rate was 83.33% for neutral, 85.72% for happy, 85.71% for sad, and 89.29% for the high mental workload. For the model evaluation of feature weighting, the minimum value of the right foot in the x- and z-axis, cadence, and stride length were ranked as the top four features. The successful classification could help administrators be aware of employers’ mental and emotional statuses during work.

A Mental Workload Classification Method Based on GCN Modified by Squeeze-and-Excitation Residual

In some complex labor production and human–machine interactions, such as subway driving, to ensure both the efficient and rapid completion of work and the personal safety of staff and the integrity of operating equipment, the level of mental workload (MW) of operators is monitored at all times. In existing machine learning-based MW classification methods, the association information between neurons in different regions is almost not considered. To solve the above problem, a graph convolution network based on the squeeze-and-excitation (SE) block is proposed. For a raw electroencephalogram (EEG) signal, the principal component analysis (PCA) dimensionality reduction operation is carried out. After that, combined with the spatial distribution between brain electrodes, the dimensionality reduction data can be converted to graph structure data, carrying association information between neurons in different regions. In addition, we use graph convolution neural network (GCN) modified by SE residual to obtain final classification results. Here, to adaptively recalibrate channel-wise feature responses by explicitly modelling interdependencies between channels, the SE block is introduced. The residual connection can ease the training of networks. To discuss the performance of the proposed method, we carry out some experiments using the raw EEG signals of 10 healthy subjects, which are collected using the MATB-II platform based on multi-task aerial context manipulation. From the experiment results, the structural reasonableness and the performance superiority of the proposed method are verified. In short, the proposed GCN modified by the SE residual method is a workable plan of mental workload classification.

Driver's mental workload classification using physiological, traffic flow and environmental factors

During the dynamic driving process, classification of mental workload for drivers is a complex task due to multiple factors, including human, vehicle, road, and the environment. Many current studies focus on the impacts of a single factor, such as physiological signal, lane changing, and so on, but these results are usually unsatisfactory due to the incapacity to account for other factors. In response, this work proposes a multi-factor quantification and analysis method to classify the driver's mental workload by incorporating physiological signal, traffic flow, and environment. For physiological signals, we use sensors to gather the driver's Electrocardiogram (ECG) and Electrodermal Activity (EDA). In order to quantify the traffic flow and environment, such as traffic volume, space headway, and weather. We use detection algorithms in images from the car's front and rear driving recorders in the real-world driving experiment on the expressway. Furthermore, we can estimate the distance between surrounding vehicles and the experimental vehicle using the detected car coordinates and camera parameters. This work then applies deep learning algorithms to classify the front driving recorder images into several weather classifications. Finally, the quantified multi-factor data can be utilized to classify the driver's mental workload. Drivers, on the other hand, annotate the data using the NASA Task Load Index Scale's results (NASA-TLX). In our experiments, we compare the classification results of single-factor and multi-factor data with three machine learning algorithms: neural networks (NN), support vector machines (SVM), and random forests (RF). The results demonstrate that for traffic volume and space headway from drive recorders, the detection performance of traffic volume in a monocular camera can reach up to 87.3%. Within sight distance, the accuracy of space headway can reach up to 87.1%. The RF algorithm achieves an outstanding classification result with strong stability by incorporating ECG, EDA, traffic flow, and environmental factors, and its classification accuracy achieves 97.8%. These findings further demonstrate the efficacy of the proposed multi-factor quantification and analysis method.

Research on Pilots ’ Mental Workload Classification in Simulated Flight

Abstract The problem of human-computer interaction mental workload in flight driving has great reference value for the prevention of safety hazards in aviation driving. This paper analyzes and studies the classification method of mental workload in flight driving by designing different simulated flight experiment tasks. This study uses a combination of EEG signals and subjective evaluation, through the use of convolutional neural networks and long short-term memory network method of combining EEG signals for research and analysis. The accuracy of EEG signal classification is as high as 94.9 %. NASA-TLX evaluation results show that there is a positive correlation between task load difficulty and evaluation score. The results show that the combination of convolutional neural network and long short-term memory network is suitable for pilots ’ mental workload classification. This study has important practical significance for flight accidents caused by pilots ’ mental workload.

Effects of Personalized Cognitive Training Using Mental Workload Monitoring on Executive Function in Older Adults With Mild Cognitive Impairment.

Although a variety of cognitive training has been performed, its optimally personalized delivery is still unknown. This study established the mental workload classification model using a convolutional neural network based on functional near-infrared spectroscopy-derived data. The dorsolateral prefrontal cortex (DLPFC) while thirty individuals with mild cognitive impairment (MCI) performed spatial working memory testing was found to be a considerable indicator to classify 3 levels of mental workload with an accuracy of over 86%. In the next step, forty subjects with MCI were randomly allocated into the experimental group (EG) that received cognitive training with mental workload-based difficulty adjustment or the control group (CG) that received conventional cognitive training. To compare both groups, the Trail Making Test Part B (TMT-B) and hemodynamic responses in the DLPFC during the TMT-B were measured. After the 16 training sessions, the EG subjects achieved a greater improvement in the TMT-B than the CG subjects (p < 0.05). Also, the EG subject showed a significantly lower DLPFC activity during the TMT-B than the CG subject (p < 0.05). In sum, the EG subjects better performed executive function with lower energy from the DLPFC. These findings imply that the importance of mental workload monitoring to provide personalized cognitive training.

EEG/fNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning.

There is high demand for techniques to estimate human mental workload during some activities for productivity enhancement or accident prevention. Most studies focus on a single physiological sensing modality and use univariate methods to analyse multi-channel electroencephalography (EEG) data. This paper proposes a new framework that relies on the features of hybrid EEG–functional near-infrared spectroscopy (EEG–fNIRS), supported by machine-learning features to deal with multi-level mental workload classification. Furthermore, instead of the well-used univariate power spectral density (PSD) for EEG recording, we propose using bivariate functional brain connectivity (FBC) features in the time and frequency domains of three bands: delta (0.5–4 Hz), theta (4–7 Hz) and alpha (8–15 Hz). With the assistance of the fNIRS oxyhemoglobin and deoxyhemoglobin (HbO and HbR) indicators, the FBC technique significantly improved classification performance at a 77% accuracy for 0-back vs. 2-back and 83% for 0-back vs. 3-back using a public dataset. Moreover, topographic and heat-map visualisation indicated that the distinguishing regions for EEG and fNIRS showed a difference among the 0-back, 2-back and 3-back test results. It was determined that the best region to assist the discrimination of the mental workload for EEG and fNIRS is different. Specifically, the posterior area performed the best for the posterior midline occipital (POz) EEG in the alpha band and fNIRS had superiority in the right frontal region (AF8).

Classification of mental workload in Human-robot collaboration using machine learning based on physiological feedback

The development of the industry phase from 4.0 into 5.0 is shifting the focus from technology-driven to value-driven to empower and engage humans in the work environment with greater influence and responsibility. This means that it is necessary to know how humans perceive collaborative work, which resulted from a shift from traditional work. Capturing the physiological conditions of humans in support of device development has shown promise in terms of the flexibility and broader scope of data in the wearable device system. In this study, the human mental workload is evaluated via physiological feedback using wearable sensors in a quantitative approach to recognize the human workload perception in the Human–robot collaboration (HRC). The physiological feedback is used to evaluate the mental workload with a classification approach utilizing supervised machine learning algorithms. The results show that, by using a random forest algorithm which selects features according to information gain, the accuracy of classifying the mental workload can be as high as 94% in the testing phase. Thus, the mental workload perceived by the operator can be estimated from physiological parameters. This finding can support Industry 5.0 applications in human–robot collaboration with a human-centric approach.

Sensitive Channel Selection for Mental Workload Classification

Mental workload (MW) assessment has been widely studied in various human–machine interaction tasks. The existing researches on MW classification mostly use non-invasive electroencephalography (EEG) caps to collect EEG signals and identify MW levels. However, the activation region of the brain stimulated by MW tasks is not the same for every subject. It may be inappropriate to use EEG signals from all electrode channels to identify MW. In this paper, an EEG rhythm energy heatmap is first established to visually show the change trends in the energy of four EEG rhythms with time, EEG channels and MW levels. It can be concluded from the presented heatmaps that this change trend varies with subjects, rhythms and channels. Based on the analysis, a double threshold method is proposed to select sensitive channels for MW assessment. The EEG signals of personalized selected channels, named positive sensitive channels (PSCs) and negative sensitive channels (NSCs), are used for MW classification using the Support Vector Machine (SVM) algorithm. The results show that the selection of personalized sensitive channels generally contributes to improving the performance of MW classification.

Evaluation of drivers' mental workload based on multi-modal physiological signals

Accurately assessing the driver's mental workload is of great significance to reduce the traffic accidents caused by the driver's mental overload. This study aims to evaluate drivers' mental workload in simulated typical driving scenarios, with <italic>N</italic>-back cognitive tasks used to manipulate varied levels of task difficulty. We collect data on multi-modal physiological signals including electroencephalogram (EEG), electrocardiogram (ECG), and electrodermal activity (EDA) signals, and subjective mental load of the National Aeronautics and Space Administration task load index (NASA_TLX) during the task completion process of the driver in the experiment, and propose a series of mental workload classification models based on feature analysis and pattern recognition of the multi-modal physiological signals. These classification models are verified by machine learning algorithms of random forest, decision tree and <italic>k</italic>-nearest neighbor models. The results show that the accuracy of classification models varies with different modalities of physiological signals. EEG-based classification models yield the highest accuracy among single-modal classification models, followed by EDA-based and ECG-based models. Multi-modal-based classification models generally perform better than single-modal classification models. The random forest classification algorithm based on three-modal physiological signals of EEG, ECG and EDA has the highest classification accuracy.