Multi-Attribute Task Battery-II Research Articles

Verification and Validation of Cognitive Workload Models for Adaptive Automation Tasks

Technological advances that seek to address future operational challenges abound. While advanced capabilities are being developed, there is an important space for human design considerations, including cognitive workload. One proposed solution to improve cognitive workload management is adaptive automation (AA). This research used a novel, model-based approach to assess the impacts of AA on cognitive workload. This assessment modeled the tasks in NASA’s Multi-attribute Task Battery-II (MATB-II) using the Improved Performance Research Integration Tool (IMPRINT). The effort sought to investigate the relationship of cognitive workload, situation awareness, and performance through three human-in-the-loop studies with 120 participants using MATB-II. The research also attempted to validate cognitive workload models from IMPRINT. The IMPRINT models were representative of the MATB tasks with statistically significant differences between workload conditions, which mirrored the models’ predictions. The results demonstrated that AA system task models can be developed using IMPRINT to provide design recommendations.

Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study.

While the term task load (TL) refers to external task demands, the amount of work, or the number of tasks to be performed, mental workload (MWL) refers to the individual's effort, mental capacity, or cognitive resources utilized while performing a task. MWL in multitasking scenarios is often closely linked with the quantity of tasks a person is handling within a given timeframe. In this study, we challenge this hypothesis from the perspective of electroencephalography (EEG) using a deep learning approach. We conducted an EEG experiment with 50 participants performing NASA Multi-Attribute Task Battery II (MATB-II) under 4 different task load levels. We designed a convolutional neural network (CNN) to help with two distinct classification tasks. In one setting, the CNN was used to classify EEG segments based on their task load level. In another setting, the same CNN architecture was trained again to detect the presence of individual MATB-II subtasks. Results show that, while the model successfully learns to detect whether a particular subtask is active in a given segment (i.e., to differentiate between different subtasks-related EEG patterns), it struggles to differentiate between the two highest levels of task load (i.e., to distinguish MWL-related EEG patterns). We speculate that the challenge comes from two factors: first, the experiment was designed in a way that these two highest levels differed only in the quantity of work within a given timeframe; and second, the participants' effective adaptation to increased task demands, as evidenced by low error rates. Consequently, this indicates that under such conditions in multitasking, EEG may not reflect distinct enough patterns to differentiate higher levels of task load.

Investigating Human Physiological Responses to Work-Related Stress

Studies have shown that work-related stress is one of the causes of employee burnout, fatigue, and cognitive dysfunction, among other negative effects. Physiological features have been used to investigate stress, but more knowledge is needed in understanding physiological indicators of work-related stress. Moreover, to the best of our knowledge, no study is available that integrates both pupillometry and heart rate in investigating work-related stress. We, therefore, utilized task-evoked pupillary response (TEPR) from pupillometry and heart rate (HR), in the assessment of physiological responses of 32 subjects during the performance of Multi-Attribute Task Battery-II consisting of working baseline and stress conditions. A comparison of results of both conditions showed that TEPR and mean HR significantly increased during stress condition, as compared to working baseline condition. These results are attributed to the work related-stressors integrated in our study, thereby bolstering the applicability of heart rate and pupillometry in studies of work-related stress.

Diurnal effects of dynamic lighting on alertness, cognition, and mood of mentally fatigued individuals in a daylight deprived environment

This study investigated the non-image forming (NIF) effects of dynamic light on alertness, cognitive performance and mood on mental fatigue, as well as moderation of task difficulty in lighting perception. Sixteen participants completed a psychomotor vigilance test (PVT) and two complex cognitive tasks (multi-attribute task battery-II (MATB-II) and n-back) with low, medium and high difficulty levels under dynamic (4000 to 12 000 K, melanopic equivalent daylight illuminance (EDI) = 224 lx to 420 lx) and static (4000 K, melanopic EDI = 224 lx) lighting with illuminance set at 500 lx on the desk. Psychological, behavioural, biochemical and electrophysiological responses to light were assessed. The results showed that exposure to dynamic light had significant benefits on subjective sleepiness, positive mood and task performance, which may be moderated by task difficulty. The intervention effect of dynamic light on fatigue was also found during an electrophysiological activity due to increase in the amount of attentional resources allocated to the tasks by dynamic light, but was not observed in biochemical measures.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation.

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

Adaptive Filtering for Improved EEG-Based Mental Workload Assessment of Ambulant Users.

Recently, due to the emergence of mobile electroencephalography (EEG) devices, assessment of mental workload in highly ecological settings has gained popularity. In such settings, however, motion and other common artifacts have been shown to severely hamper signal quality and to degrade mental workload assessment performance. Here, we show that classical EEG enhancement algorithms, conventionally developed to remove ocular and muscle artifacts, are not optimal in settings where participant movement (e.g., walking or running) is expected. As such, an adaptive filter is proposed that relies on an accelerometer-based referential signal. We show that when combined with classical algorithms, accurate mental workload assessment is achieved. To test the proposed algorithm, data from 48 participants was collected as they performed the Revised Multi-Attribute Task Battery-II (MATB-II) under a low and a high workload setting, either while walking/jogging on a treadmill, or using a stationary exercise bicycle. Accuracy as high as 95% could be achieved with a random forest based mental workload classifier with ambulant users. Moreover, an increase in gamma activity was found in the parietal cortex, suggesting a connection between sensorimotor integration, attention, and workload in ambulant users.

Effects of Automation Reliability and Trust on System Monitoring Performance in Simulated Flight Tasks

The current study was designed to empirically examine the effects of trust and automation reliability on multi-tasking performance in a simulated cockpit setting using the Multi-Attribute Task Battery II (MATB-II). The MATB-II simulates tasks often performed by pilots in-flight, tasking the operator with attending to automated systems and correcting errors when they inevitably occur. Over the course of three 30-minute trials, two levels of automation reliability were presented in the current study (R50% and R90%). It was hypothesized that automation reliability and trust would affect both workload and performance in this multi-tasking environment. Results indicated that reliability significantly affected monitoring performance on the MATB-II. More reliable automation resulted in poorer monitoring performance, while trust appeared to have little impact. These results provide further evidence for how operators trust and interact with automation, a topic that is relevant to the implementation of automated systems in a variety of human-machine systems such as aviation.

Quantitative modeling of user performance in multitasking environments

Multitasking is one of the most important skills required for human operators to perform highly-complex and safety-critical jobs. This study proposed and validated a quantitative model for the study of user performance improvement in a multitasking environment. The Multi-Attribute Task Battery-II (MATB-II) was used in the experiments as a multitasking platform. The proposed model included quantification of stimuli from each MATB-II subtask as baud rate (bits per second), selection of task difficulty and task weight, as well as the rearrangement of task weights. This research followed a two-phase experimental approach. The first phase applied the proposed model and identified a performance baseline for each individual in a multitasking environment, MATB-II. The second phase validated the proposed model using a rearranged set of multitasks for each individual. Individual differences in working memory capacity (WMC) have been estimated as predictors of varying cognitive abilities. This study also investigated the relationship between WMC, task difficulty, and multitasking performance. Significant improvement of user performance was found after the rearrangement of tasks based on the proposed approach. This research provides a framework to quantitatively evaluate multitasking systems and improve human performance in order to understand the interaction between systems and human operators.

Impact of Auditory Interference on Automated Task Monitoring and Workload

The purpose of this research was to explore the effects of auditory distraction and criticality on pilots’ flight accuracy and workload, along with attention allocation with different reliability levels on the Multi- Attribute Task Battery II (MATB-II) simulator. The MATB-II involves attending to multiple systems and correcting errors made by the automation, which will be presented at multiple levels of reliability (50%, 70%, 90%) and distractor tasks. The presence of a complex communication task before or concurrent with the main task negatively impacts accuracy of automation failure detection (Stader, 2014). It was hypothesized that reliability level, as well as the presence of an auditory distractor task would affect human monitoring performance and workload. The data indicated distraction could affect the accuracy of pilot’s multi-tasking abilities, as well as flight pattern proficiencies and workload. This information would be applicable to cockpit protocol and ensure a more efficient flight.

A colored petri nets based workload evaluation model and its validation through Multi-Attribute Task Battery-II

This paper proposed a colored petri nets based workload evaluation model. A formal interpretation of workload was firstly introduced based on the process that reflection of petri nets components to task. A petri net based description of Multiple Resources theory was given by comprehending it from a new angle. A new application of VACP rating scales named V/A-C-P unit, and the definition of colored transitions were proposed to build a model of task process. The calculation of workload mainly has the following four steps: determine token's initial position and values; calculate the weight of directed arcs on the basis of the rules proposed; calculate workload from different transitions, and correct the influence of repetitive behaviors. Verify experiments were carried out based on Multi-Attribute Task Battery-II software. Our results show that there is a strong correlation between the model values and NASA -Task Load Index scores (r=0.9513). In addition, this method can also distinguish behavior characteristics between different people.

Assessment of the interaction of hyperbaric N2, CO2, and O2 on psychomotor performance in divers.

Diving narcosis results from the complex interaction of gases, activities, and environmental conditions. We hypothesized that these interactions could be separated into their component parts. Where previous studies have tested single cognitive tasks sequentially, we varied inspired partial pressures of CO2, N2, and O2 in immersed, exercising subjects while assessing multitasking performance with the Multi-Attribute Task Battery II (MATB-II) flight simulator. Cognitive performance was tested under 20 conditions of gas partial pressure and exercise in 42 male subjects meeting U.S. Navy age and fitness profiles. Inspired nitrogen (N2) and oxygen (O2) partial pressures were 0, 4.5, and 5.6 ATA and 0.21, 1.0, and 1.22 ATA, respectively, at rest and during 100-W immersed exercise with and without 0.075-ATA CO2 Linear regression modeled the association of gas partial pressure with task performance while controlling for exercise, hypercapnic ventilatory response, dive training, video game frequency, and age. Subjects served as their own controls. Impairment of memory, attention, and planning, but not motor tasks, was associated with N2 partial pressures >4.5 ATA. Sea level O2 at 0.925 ATA partially rescued motor and memory reaction time impaired by 0.075-ATA CO2; however, at hyperbaric pressures an unexpectedly strong interaction between CO2, N2, and exercise caused incapacitating narcosis with amnesia, which was augmented by O2 Perception of narcosis was not correlated with actual scores. The relative contributions of factors associated with diving narcosis will be useful to predict the effects of gas mixtures and exercise conditions on the cognitive performance of divers. The O2 effects are consistent with O2 narcosis or enhanced O2 toxicity.

Individual Differences in Dynamic Multitasking Performance

The current study examined whether individual differences in memory, attention, and visuospatial ability are significant predictors of multitasking ability (multitasking performance baseline level) and multitasking adaptability (the capacity to adapt to dynamic changes in task demands). Participants were administered a neuropsychological battery to measure individual differences in cognitive abilities. Then, participants performed the Multi-Attribute Task Battery-II (MATB) comprising of three workload conditions. Results indicated that participants scoring higher on measures of attention performed better on the MATB during the baseline condition (ability). However, higher scores on measures of delayed memory predicted better performance during more demanding MATB conditions (adaptability), while visuospatial ability predicted worsened performance during more demanding MATB conditions. Additionally, higher global neuropsychological functioning predicted better MATB performance during all conditions.

Effects of Working Memory Capacity, Task Switching, and Task Difficulty on Multitasking Performance

Multitasking requires human operators to handle the demands of multiple tasks through task switching at the same time and this ability is required in many jobs. Previous studies showed that different levels of working memory capacity (WMC) and task switching abilities can lead to differences on multitasking performance. With increased complexity of tasks, maintaining task performance is challenging. This study sought to find the relations of WMC, task switching, task difficulty, and multitasking performance. Multi-Attribute Task Battery II (MATB-II) was employed in this study as a platform to assess multitasking. Automated OSPAN and Trail Making Tasks (TMT) were used to assess WMC and the task switching ability, respectively. Results indicated that there were significant effects of these three parameters on multitasking performance. Other dimensions of multitasking performance will be addressed in future studies.

Using Pupillometry to Indicate the Cognitive Redline

Physiological measures, which are influenced by the arousal of the autonomous nervous system, have been studied as indicators of extreme levels of mental workload that approach or exceed the cognitive “redline”, the point at which task demand exceeds the supply of cognitive resources. In response to increasing task demands, measures such as heart rate variability show asymptotic patterns in arousal that are consistent with plateau patterns in subjective self-reported measures. This suggests potential to use physiological indicators in real time to predict when an operator is at increased risk of cognitive overload. Expanding on prior work, the current study examined pupil diameter as a new potential indicator of the cognitive redline in a multitask environment created with the Multi-Attribute Task Battery-II (MATB-II). Results showed that pupil diameter is sensitive to imposed mental workload and exhibits a similar asymptotic pattern that may provide another potential real-time indicator of the cognitive redline.

Individual Differences in Multitasking Performance

The present study examined whether personality characteristics and general intelligence predict multitasking performance. The Multi-Attribute Task Battery-II was used to assess multitasking performance. Personality factors included the Big Five, openness, conscientiousness, extraversion, agreeableness, and neuroticism. The results indicated scores on general intelligence predict performance on the tracking task of the Multi-Attribute Task Battery-II, where higher scores of general intelligence predicted improved tracking performance. Additionally, conscientiousness and neuroticism were found to predict worsened performance on the resource management task of the Multi-Attribute Task Battery-II. Furthermore, agreeableness was found to predict perceived workload on the mental demand subscale of the Workload Rating Scale.

Physiological Indicators of the Cognitive Redline

As suggested in recent research, some physiological measures such as breath rate, heart rate, heart rate variability, electroencephalography, and skin conductance response, can be used to indicate aspects of mental workload. Some of these physiological measures are insensitive to added mental workload at a point referred to as the cognitive redline. This redline is not well indicated in the literature, but is generally defined by patterns of performance decrements. The present study involved the use of the Multi-Attribute Task Battery-II (MATB-II) to systematically manipulate imposed multitask workload, and investigate the validity of online measures of heart rate variability (HRV) to indicate the cognitive redline, as inferred by performance and subjective workload measures.

Effects of Display and Task Features on System Monitoring Performance in the Original Multi-Attribute Task Battery and MATB-II

This study was designed to empirically examine equipment and system monitoring task features of the new Multi-Attribute Task Battery-I I (MATB-II) and the original version (MATB). It also was intended to examine the effects of adaptive task allocation on system monitoring performance, as found in previous studies. In addition, two different computer displays (CRT and LCD) were used to test performance differences between the two systems in order to add historical fidelity to the study results. Sixty participants were randomly assigned to one of four experimental groups (MATB or MATB-II both on CRT or LCD), all receiving a return to manual control of the system monitoring task at mid experiment. A fifth group using the MATB on CRT with no task allocation was the control. Measures used to compare the two systems included percent of automation failures detected by participants in the system monitoring task and subjective workload. Results failed to show significance for the adaptive task allocation manipulation. However, significant differences in monitoring performance were found between the two systems. Implications for design of display and equipment features and future research are discussed.