Multi-Attribute Task Battery Research Articles

Examining mental workload based on multiple physiological signals: review of the multi-attribute task battery (MATB) technique

Examining mental workload based on multiple physiological signals: review of the multi-attribute task battery (MATB) technique

MATB for assessing different mental workload levels.

Multi-Attribute Task Battery (MATB) is a computerized flight simulator for aviation-related tasks, suitable for non-pilots and available in many versions, including open source. MATB requires the individual or simultaneous execution of 4 sub-tasks: system monitoring (SYSMON), tracking (TRACK), communications (COMM), and resource management (RESMAN). Fully customizable, the design of test duration, number of sub-tasks used, event rates, response times and overlap, create different levels of mental load. MATB can be combined with an additional auditory attention (Oddball) task, or with physiological constraints (i.e., sleep loss, exercise, hypoxia). We aimed to assess the main characteristics of MATB design for assessing the response to different workload levels. We identified and reviewed 19 articles for which the effects of low and high workload were analyzed. Although MATB has shown promise in detecting performance degradation due to increase workload, studies have yielded conflicting or unclear results regarding MATB configurations. Increased event rates, number of sub-tasks (multitasking), and overlap are associated with increased perceived workload score (ex. NASA-TLX), decreased performance (especially tracking), and neurophysiological responses, while no effect of time-on-task is observed. The median duration used for the test is 20min (range 12-60) with a level duration of 10min (range 4-15). To assess mental workload, the median number of stimuli is respectively 3 events/min (range 0.6-17.2) for low, and 23.5 events/min (range 9-65) for high workload level. In this review, we give some recommendations for standardization of MATB design, configuration, description and training, in order to improve reproducibility and comparison between studies, a challenge for the future researches, as human-machine interaction and digital influx increase for pilots. We also open the discussion on the possible use of MATB in the context of aeronautical/operational constraints in order to assess the effects combined with changes in mental workload levels. Thus, with appropriate levels of difficulty, MATB can be used as a suitable simulation tool to study the effects of changes on the mental workload of aircraft pilots, during different operational and physiological constraints.

Evaluating gender-based effectiveness within multi-attribute task battery studies

The prevalent notion that females outperform males in multitasking is challenged by scrutinizing the cognitive abilities of each gender across various demand scenarios (Buser & Peter, 2012; Lui et al., 2020; Szameitat et al., 2015). Investigating cognitive performance at two levels of workload defined as high- and low-demand situations, this research delves into task-specific variations, cognitive strategies, and adaptability to multitasking challenges. Sixty participants in total, with thirty participants in each gender group, engaged in tasks encompassing resource management, tracking, monitoring, and communication using the U.S. Army Aeromedical Research Laboratory's Multi- Attribute Task Battery simulation software. Statistical variance analysis demonstrated notable task discrepancies across different difficulty levels (DL). However, the gender-based analysis indicated non-significant differences in given tasks, except communication tasks. Findings indicate that men tend to perform well in tasks requiring communication and target tracking, benefiting from auditory cues, while women excel in fuel management, displaying strong decision-making abilities. In assessments of visual and spatial perception, females outperformed males at low difficulty levels, whereas males showed greater proficiency at high difficulty levels. This paper advocates for a balanced understanding and approach of cognitive abilities in diverse scenarios.

Priority influences task selection decisions in multi-task management

The role of task priority on task selection in multi-task management is unclear based on prior work, leading to a common finding of ‘priority neglect’. However, properties such as urgency and conflict may influence whether operators weigh priority in their decision. We examined the role of instructed task prioritization, bolstered by more urgent and conflicting conditions, on how operators select among emergent, concurrent tasks when multitasking. Using the Multi-Attribute Task Battery (MATB) multitasking platform we tested both an auditory communications task and a manual tracking task as the priority tasks. Results showed that instructed priority significantly increased target task selection under the conflicting task conditions for both tasks. Urgency itself may modulate whether instructions to prioritize affect task selection choices when multitasking, and therefore counter to prior results instructions may yet be useful for helping operators select a higher priority task under conflict, a generalizable effect to be further explored.

$\beta$ -wave-based exploration of sensitive EEG features and classification of situation awareness

AbstractThe purpose of this study was to explore the electroencephalogram (EEG) features sensitive to situation awareness (SA) and then classify SA levels. Forty-eight participants were recruited to complete an SA standard test based on the multi-attribute task battery (MATB) II, and the corresponding EEG data and situation awareness global assessment technology (SAGAT) scores were recorded. The population with the top 25% of SAGAT scores was selected as the high-SA level (HSL) group, and the bottom 25% was the low-SA level (LSL) group. The results showed that (1) for the relative power of $\beta$ 1 (16–20Hz), $\beta$ 2 (20–24Hz) and $\beta$ 3 (24–30Hz), repeated measures analysis of variance (ANOVA) in three brain regions (Central Central-Parietal, and Parietal) × three brain lateralities (left, midline, and right) × two SA groups (HSL and LSL) showed a significant main effect for SA groups; post hoc comparisons revealed that compared with LSL, the above features of HSL were higher. (2) for most ratio features associated with $\beta$ 1 ∼ $\beta$ 3, ANOVA also revealed a main effect for SA groups. (3) EEG features sensitive to SA were selected to classify SA levels with small-sample data based on the general supervised machine learning classifiers. Five-fold cross-validation results showed that among the models with easy interpretability, logistic regression (LR) and decision tree (DT) presented the highest accuracy (both 92%), while among the models with hard interpretability, the accuracy of random forest (RF) was 88.8%, followed by an artificial neural network (ANN) of 84%. The above results suggested that (1) the relative power of $\beta$ 1 ∼ $\beta$ 3 and their associated ratios were sensitive to changes in SA levels; (2) the general supervised machine learning models all exhibited good accuracy (greater than 75%); and (3) furthermore, LR and DT are recommended by combining the interpretability and accuracy of the models.

Studying the Role of Visuospatial Attention in the Multi-Attribute Task Battery II

Task batteries mimicking user tasks are of high heuristic value. Supposedly, they measure individual human aptitude regarding the task in question. However, less is often known about the underlying mechanisms or functions that account for task performance in such complex batteries. This is also true of the Multi-Attribute Task Battery (MATB-II). The MATB-II is a computer display task. It aims to measure human control operations on a flight console. Using the MATB-II and a visual-search task measure of spatial attention, we tested if capture of spatial attention in a bottom-up or top-down way predicted performance in the MATB-II. This is important to understand for questions such as how to implement warning signals on visual displays in human–computer interaction and for what to practice during training of operating with such displays. To measure visuospatial attention, we used both classical task-performance measures (i.e., reaction times and accuracy) as well as novel unobtrusive real-time pupillometry. The latter was done as pupil size covaries with task demands. A large number of analyses showed that: (1) Top-down attention measured before and after the MATB-II was positively correlated. (2) Test-retest reliability was also given for bottom-up attention, but to a smaller degree. As expected, the two spatial attention measures were also negatively correlated with one another. However, (3) neither of the visuospatial attention measures was significantly correlated with overall MATB-II performance, nor with (4) any of the MATB-II subtask performance measures. The latter was true even if the subtask required visuospatial attention (as in the system monitoring task of the MATB-II). (5) Neither did pupillometry predict MATB-II performance, nor performance in any of the MATB-II’s subtasks. Yet, (6) pupil size discriminated between different stages of subtask performance in system monitoring. This finding indicated that temporal segregation of pupil size measures is necessary for their correct interpretation, and that caution is advised regarding average pupil-size measures of task demands across tasks and time points within tasks. Finally, we observed surprising effects of workload (or cognitive load) manipulation on MATB-II performance itself, namely, better performance under high- rather than low-workload conditions. The latter findings imply that the MATB-II itself poses a number of questions about its underlying rationale, besides allowing occasional usage in more applied research.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments.

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings. Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken. Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05). This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study.

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

The use of multi-attribute task battery in mental workload studies: A scoping review

Multi-Attribute Task Battery (MATB) is a software that has been arguably utilized in many ergonomics/human factors studies, including in the topic of mental workload. However, the use of this well-known program in diverse investigations has not yet been systematically tracked. Furthermore, it may be argued that a critical appraisal of MATB is urgently needed so that future researchers and users can take several crucial factors into account when planning a study or experiment using MATB. The aim of this paper is to comprehensively identify and review the use of MATB software in published studies. This aim might be accomplished by achieving two goals: (1) systematic discovery of published papers in literature databases and (2) categorization of research according to pertinent topics. In this paper, thirty-one articles were included for analysis after carefully screening for their eligibility. Our scoping review finds that MATB is a beneficial program for creating multitasking environments in general, with aviation being the area where it has been used the most. The program was also extensively used for studies on mental workload, especially by producing various stimuli that ultimately result in varying degrees of task demand or difficulty. Moreover, to successfully use MATB, researchers must be aware of a few operational issues and criticisms.

Evaluation of a headphones-fitted EEG system for the recording of auditory evoked potentials and mental workload assessment

Advancements in portable neuroimaging technologies open up new opportunities to gain insight into the neural dynamics and cognitive processes underlying day-to-day behaviors. In this study, we evaluated the relevance of a headphone- mounted electroencephalogram (EEG) system for monitoring mental workload. The participants (N = 12) were instructed to pay attention to auditory alarms presented sporadically while performing the Multi-Attribute Task Battery (MATB) whose difficulty was staged across three conditions to manipulate mental workload. The P300 Event-Related Potentials (ERP) elicited by the presentation of auditory alarms were used as probes of attentional resources available. The amplitude and latency of P300 ERPs were compared across experimental conditions. Our findings indicate that the P300 ERP component can be captured using a headphone-mounted EEG system. Moreover, neural responses to alarm could be used to classify mental workload with high accuracy (over 80%) at a single-trial level. Our analyses indicated that the signal-to-noise ratio acquired by the sponge-based sensors remained stable throughout the recordings. These results highlight the potential of portable neuroimaging technology for the development of neuroassistive applications while underscoring the current limitations and challenges associated with the integration of EEG sensors in everyday-life wearable technologies. Overall, our study contributes to the growing body of research exploring the feasibility and validity of wearable neuroimaging technologies for the study of human cognition and behavior in real-world settings.

Evidence for modulation of EEG microstates by mental workload levels and task types.

Electroencephalography (EEG) microstate analysis has become a popular tool for studying the spatial and temporal dynamics of large-scale electrophysiological activities in the brain in recent years. Four canonical topographies of the electric field (classes A, B, C, and D) have been widely identified, and changes in microstate parameters are associated with several psychiatric disorders and cognitive functions. Recent studies have reported the modulation of EEG microstate by mental workload (MWL). However, the common practice of evaluating MWL is in a specific task. Whether the modulation of microstate by MWL is consistent across different types of tasks is still not clear. Here, we studied the topographies and dynamics of microstate in two independent MWL tasks: NBack and the multi-attribute task battery (MATB) and showed that the modulation of MWL on microstate topographies and parameters depended on tasks. We found that the parameters of microstates A and C, and the topographies of microstates A, B, and D were significantly different between the two tasks. Meanwhile, all four microstate topographies and parameters of microstates A and C were different during the NBack task, but no significant difference was found during the MATB task. Furthermore, we employed a support vector machine recursive feature elimination procedure to investigate whether microstate parameters were suitable for MWL classification. An averaged classification accuracy of 87% for within-task and 78% for cross-task MWL discrimination was achieved with at least 10 features. Collectively, our findings suggest that topographies and parameters of microstates can provide valuable information about neural activity patterns with a dynamic temporal structure at different levels of MWL, but the modulation of MWL depends on tasks and their corresponding functional systems. Moreover, as a potential indicator, microstate parameters could be used to distinguish MWL.

Effects of adaptation accuracy and magnitude in affect-aware difficulty adaptation for the multi-attribute task battery

In affective computing, classification algorithms are used to recognize users’ psychological states and adapt tasks to optimize user experience. However, classification is never perfect, and the relationship between adaptation accuracy and user experience remains understudied. It is also unclear whether the adaptation magnitude (‘size’ of action taken to influence user states) influences effects of adaptation accuracy. To evaluate impacts of adaptation accuracy (appropriate vs. inappropriate actions) and magnitude on user experience, we conducted a ‘Wizard of Oz’ study where 112 participants interacted with the Multi-Attribute Task Battery over three 11-minute intervals. An adaptation accuracy (50 % to 80 %) was preassigned for the first 11-minute interval, and accuracy increased by 10 % in each subsequent interval. Task difficulty changed every minute, and participant preferences for difficulty changes were assessed at the same time. Adaptation accuracy was artificially induced by fixing the percentage of times the difficulty changes matched participant preferences. Participants were also randomized to two magnitude conditions, with difficulty modified by 1 (low) or 3 (high) levels each minute. User experience metrics were assessed after each interval.Analysis with latent growth models offered support for linear increases in user experience across increasing levels of adaptation accuracy. For each 10 % gain in accuracy, results indicate a 1.3 (95 % CI [.35, 2.20]) point increase in NASA Task Load Index scores (range 6–60), a 0.40 (95 % CI [.18, 0.57]) increase in effort/importance (range 2–14), and 0.48 (95 % CI [.24, 0.72]) increase in perceived competence (range 2–14). Furthermore, the effect of accuracy on Task Load Index scores was modulated by adaptation magnitude. No effects were observed for interest/enjoyment or pressure/tension. By providing quantitative estimates of effects of adaptation accuracy on user experience, the study provides guidelines for researchers and developers of affect-aware technologies. Furthermore, our methods could be adapted for use in other affective computing scenarios.

Examining post-error performance in a complex multitasking environment

Previous work on indices of error-monitoring strongly supports that errors are distracting and can deplete attentional resources. In this study, we use an ecologically valid multitasking paradigm to test post-error behavior. It was predicted that after failing an initial task, a subject re-presented with that task in conflict with another competing simultaneous task, would more likely miss their response opportunity for the competing task and stay ‘tunneled’ on the initially errored task. Additionally, we predicted that an error’s effect on attention would dissipate after several seconds, making error cascades less likely when subsequent conflict tasks are delayed. A multi-attribute task battery was used to present tasks and collect measures of both post-error and post-correct performance. Results supported both predictions: post-error accuracy on the competing task was lower compared to post-correct accuracy, and error-proportions were higher at shorter delays, dissipating over time. An exploratory analysis also demonstrated that following errors (as opposed to post-correct trials), participants clicked more on the task panel of the initial error regardless of delay; this continued task-engagement provides preliminary support for errors leading to a cognitive tunneling effect.

The effect of punishment and feedback on correcting erroneous behavior

Introduction: Understanding the consequences of non-punitive sanctions and feedback for nonintentional deviations (i.e., errors) is important to effective safety policy. This study aims to address a lack of research on the effects of punishment and feedback on correcting erroneous behavior in the context of multitasking. Method: A Multi-Attribute Task Battery (MATB-II) was employed to simulate the demands of aviating, an important area of applied safety. Sixty participants were randomly assigned to one of four experimental groups (no intervention, punishment, feedback, punishment + feedback) and asked to perform the MATB-II. Punishment, feedback, and punishment + feedback decreased error and increased performance, with punishment alone having the greatest effect. Results: The results highlight the need for behavioral consequences or feedback to reduce erroneous behavior. Practical Applications: From an applied perspective, these results have implications for policy and training.

Optimizing Automated Alarm Systems Through the Integration of Operator Preferences and Proactive Monitoring: Initial Testing

Efforts to optimize automated alarm systems have introduced the use of supplementary graphical proactive monitoring displays (GPMDs) that promote salient deviation detection alongside alarms. This study proposes the incorporation of newer, adaptable automation techniques, which afford operators control over an automated alarm’s reliability, in combination with GPMDs to further optimize these systems. A 2 (choice in reliability) x 2 (supplementary display) case-control match design was used to examine effects on task performance, operator mood, trust in automation, and subjective workload using an open-source version of the NASA Multi-Attribute Task Battery (OpenMATB). Results indicate that experimental participants rated the automation significantly higher on a trust in automation subscale than those in the control group. Few negative findings indicate it would be worthwhile to continue investigating the proposed system adaptations to determine if there are benefits to be gained from their application in realworld scenarios where risk of failure is especially dangerous.

The effect of two-stage warning system on human performance along with different takeover strategies

Warning stages and takeover strategies are two key factors that affect the takeover process when highly automated systems fail. However, the interaction between these two factors remains unclear. This study empirically investigated the effect of warning types (two-stage warning vs. single-stage warning) and takeover strategies (interruption vs. multitasking) on human performance. Thirty-eight participants performed three tasks of the Multi-Attribute Task Battery (MATB), in which the system monitoring task was automated and adopted as the takeover task, while tracking and resource management tasks were employed as the initial tasks simultaneously. The results showed that compared with the single-stage warning system, the two-stage warning system enhanced situation awareness, reduced mental workload and improved takeover performance. However, tracking task performance declined when the two-stage warning system was used. Moreover, participants using the two-stage warning system reacted to gauge malfunctions much more quickly than participants using the single-stage warning system when the interruption strategy was adopted. However, there was no significant difference in tracking task performance between the two warning systems when the multitasking strategy was adopted. This study concluded that the two-stage warning system can benefit the takeover process but at the cost of the performance of initial tasks, and the takeover strategy plays an important role in realizing the benefits of the two-stage warning system and alleviating its costs. These findings could provide useful insights for automated system design and personnel training.

How Can One Share a User’s Activity during VR Synchronous Augmentative Cooperation?

Collaborative virtual environments allow people to work together while being distant. At the same time, empathic computing aims to create a deeper shared understanding between people. In this paper, we investigate how to improve the perception of distant collaborative activities in a virtual environment by sharing users’ activity. We first propose several visualization techniques for sharing the activity of multiple users. We selected one of these techniques for a pilot study and evaluated its benefits in a controlled experiment using a virtual reality adaptation of the NASA MATB-II (Multi-Attribute Task Battery). Results show (1) that instantaneous indicators of users’ activity are preferred to indicators that continuously display the progress of a task, and (2) that participants are more confident in their ability to detect users needing help when using activity indicators.

Cognitive Load Prediction from Multimodal Physiological Signals using Multiview Learning.

Predicting cognitive load is a crucial issue in the emerging field of human-computer interaction and holds significant practical value, particularly in flight scenarios. Although previous studies have realized efficient cognitive load classification, new research is still needed to adapt the current state-of-the-art multimodal fusion methods. Here, we proposed a feature selection framework based on multiview learning to address the challenges of information redundancy and reveal the common physiological mechanisms underlying cognitive load. Specifically, the multimodal signal features (EEG, EDA, ECG, EOG, & eye movements) at three cognitive load levels were estimated during multiattribute task battery (MATB) tasks performed by 22 healthy participants and fed into a feature selection-multiview classification with cohesion and diversity (FS-MCCD) framework. The optimized feature set was extracted from the original feature set by integrating the weight of each view and the feature weights to formulate the ranking criteria. The cognitive load prediction model, evaluated using real-time classification results, achieved an average accuracy of 81.08% and an average F1-score of 80.94% for three-class classification among 22 participants. Furthermore, the weights of the physiological signal features revealed the physiological mechanisms related to cognitive load. Specifically, heightened cognitive load was linked to amplified δ and θ power in the frontal lobe, reduced α power in the parietal lobe, and an increase in pupil diameter. Thus, the proposed multimodal feature fusion framework emphasizes the effectiveness and efficiency of using these features to predict cognitive load.

Experimental Investigation of Task Performance and Human Vigilance in Different Noise Environments

Twelve healthy male college-age students were recruited to investigate the effects of different noise exposure conditions on complex task performance and vigilance. During each noise exposure, the Multi-Attribute Task Battery (MATB) with low, medium, and high mental workloads were conducted in the order designated by the Latin square method. Meanwhile, a psychomotor vigilance test (PVT) was used to evaluate human vigilance. Heart rate variability (HRV) signals were also collected while participants performed the tasks. The generalized additive mixed-effect model (GAMM) results showed that the increased mental workload had an inverted U-shaped effect on MATB task performance. Noise exposure had no significant impact on the overall performance of MATB tasks. However, when exposed to increased noise sharpness at low mental workloads, Tracking Task (TRA) performance significantly decreased, whereas the System Monitoring Task (SYS) performance was significantly improved. In addition, higher noise sound pressure level and sharpness would impair human vigilance, which was reflected in a lower mean sample entropy of HRV and worse performance on the PVT. The results indicated that noise control in the workplace should consider both sound pressure level and sharpness.

Effects of Noise Exposure and Mental Workload on Physiological Responses during Task Execution.

Twelve healthy male students were recruited to investigate the physiological response to different noise exposure and mental workload (MW) conditions, while performing multi-attribute task battery (MATB) tasks. The experiments were conducted under three noise exposure conditions, with different sound pressure levels and sharpness. After adaptation to each noise condition, the participants were required to perform the resting test and the MATB task tests with low, medium, and high MW. The electroencephalogram (EEG), electrocardiogram (ECG), and eye movement data were obtained, during the periods when participants were in the resting and task taking state. The results showed that subjects’ physiological responses at rest were unaffected by noise exposure conditions. However, during the execution of MATB tasks, the elevated sound pressure level and increased sharpness were significantly correlated with increased mean pupil diameter and heart rate variability (HRV). These responses suggested that the human body defends itself through physiological regulation when noise causes adverse effects. If the negative effects of noise were more severe, this could damage the body’s health and result in a significant drop in task performance. The elevated mental demands led to increased stress on the subjects, which was reflected in a considerable increase in theta relative power. Either high or low MW was related with reduced saccade amplitude and a decrease in weighted task performance, indicating an inverted U-shaped relationship between workload level and work performance.