Subjective Ratings Of Mental Workload Research Articles

Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios.

Assessing drivers' mental workload is crucial for reducing road accidents. This study examined drivers' mental workload in a simulated auditory-based dual-task driving scenario, with driving tasks as the main task, and auditory-based N-back tasks as the secondary task. A total of three levels of mental workload (i.e., low, medium, high) were manipulated by varying the difficulty levels of the secondary task (i.e., no presence of secondary task, 1-back, 2-back). Multimodal measures, including a set of subjective measures, physiological measures, and behavioral performance measures, were collected during the experiment. The results showed that an increase in task difficulty led to increased subjective ratings of mental workload and a decrease in task performance for the secondary N-back tasks. Significant differences were observed across the different levels of mental workload in multimodal physiological measures, such as delta waves in EEG signals, fixation distance in eye movement signals, time- and frequency-domain measures in ECG signals, and skin conductance in EDA signals. In addition, four driving performance measures related to vehicle velocity and the deviation of pedal input and vehicle position also showed sensitivity to the changes in drivers' mental workload. The findings from this study can contribute to a comprehensive understanding of effective measures for mental workload assessment in driving scenarios and to the development of smart driving systems for the accurate recognition of drivers' mental states.

The Impact of False Alarms on Mental Workload, Stress and Performance: Implications for Medical Settings

The excessive number of auditory and false alarms (FAs) in medical settings (e.g. ICUs) negatively impact alarm response and performance on ongoing tasks. We examined the impact of two FA rates (low-40% and high-80% in a counterbalanced order repeated measures design) on 36 participants’ alarm responses with a concurrent working-memory (N-back) task. Participants made significantly more responses to FAs in the high FA condition relative to the low FA condition. An FA rate and order interaction revealed that participants who started in the low FA condition responded to fewer FAs in both the high and low FA conditions. Participants who started in the high FA condition responded to more than twice the number of FAs in the high FA condition compared to the low FA condition. Similarly, response time to correct detections (Hits) in the alarm and N-back tasks also revealed interaction effects between order and FA conditions. Participants who received the low FA condition first responded faster to true alarms in the high FA condition compared to those who received the high FA condition first. No significant main effects or interactions were observed for the number of Hits on the alarm task, response times to FAs in either task, nor for subjective ratings of mental workload or stress. Significant main and interaction effects observed on the N-back task were likely attributable to practice effects. These results underscore the need to reduce the number of FAs in medical settings and have implications for designing training paradigms to improve provider performance.

Subjective and objective assessments of mental workload for UAV operations.

Unmanned Aerial Vehicles (UAVs) have created safety problems for the publics. Assessments of the mental workload for UAV operations are essential to realize the causes of UAV accidents. To test the following hypotheses: i. mission difficulty in UAV operation affects both subjective and objective measures of mental workload; ii. mission difficulty affects number of failures in UAV operation. Fourteen male adults participated as UAV operators after attending a UAV training course. They performed four flight missions of different difficulty levels. During their flight missions, their heart rate and inter-beat interval (IBI) were collected. Upon completing each flight mission, the participants gave subjective ratings of mental workload using three commonly adopted assessment tools. The time of flight and number of failures in flight operations were also recorded. The results showed that mission difficulty affected the scores of all three assessment tools significantly. Mission difficulty also affected number of failures and IBI significantly. The scores of the three assessment tools were highly correlated (ρ= 0.7 to 0.83, p < 0.001) with one another. The results of the three subjective ratings were also consistent with that of the IBI data. High mental workload in UAV operation could lead to poor flight performance.

Measuring Mental Workload Using ERPs Based on FIR, ICA, and MARA

Mental workload is considered to be strongly linked to human performance, and the ability to measure it accurately is key for balancing human health and work. In this study, brain signals were elicited by mental arithmetic tasks of varying difficulty to stimulate different levels of mental workload. In addition, a finite impulse response (FIR) filter, independent component analysis (ICA), and multiple artifact rejection algorithms (MARAs) were used to filter event-related potentials (ERPs). Then, the data consisting of ERPs, subjective ratings of mental workload, and task performance, were analyzed through the use of variance and Spearman’s correlation during a simulated computer task. We found that participants responded faster and performed better in the easy task condition, followed by the medium and high-difficulty conditions, which verifies the validity of the ERP filtering. Moreover, larger P2 and P3 waveforms were evoked as the task difficulty increased, and a higher task difficulty elicited a more enhanced N300. Correlation analysis revealed a negative relationship between the amplitude of P3 and the subjective ratings, and a positive relationship between the P3 amplitude and accuracy. The results presented in this paper demonstrate that a combination of FIR, ICA, and MARA methods can filter ERPs in the non-invasive real-time measurement of workload. Additionally, frontocentral P2, N3, and parietal P3 components showed differences between genders. The proposed measurement of mental workload can be useful for real-time identification of mental states and can be applied to human–computer interaction in the future.

Exploring the Peak-End Effects in Air Traffic Controllers' Mental Workload Ratings.

This study examined whether professional air traffic controllers (ATCos) were subject to peak-end effects in reporting their mental workload after performing an air traffic control task, and in predicting their mental workload in future scenarios. In affective experience studies, people's evaluation of a period of experience is strongly influenced by the most intense (peak) point and the endpoint. However, whether the effects exist in mental workload evaluations made by professional operators is still not known. In Study 1, 20 ATCos performed air traffic control scenarios on high-fidelity radar simulators and reported their mental workload. We used a 2 (high peak, low peak) × 2 (high end, low end) within-subject design. In Study 2, another group of 43 ATCos completed a survey asking them to predict their mental workload given the same air traffic control scenarios. In Study 1, ATCos reported higher mental workload after completing the high-peak and the high-end scenarios. In contrast, in Study 2, ATCos predicted the peak workload effect but not the end workload effect when asked to predict their experience in dealing with the same scenarios. Peak and end effects exist in subjective mental workload evaluation, but experts only had meta-cognitive awareness of the peak effect, and not the end effect. Researchers and practitioners that use subjective workload estimates for work design decisions need to be aware of the potential impact of peak and end task demand effects on subjective mental workload ratings provided by expert operators.

Measurement and identification of mental workload during simulated computer tasks with multimodal methods and machine learning

This study attempted to multimodally measure mental workload and validate indicators for estimating mental workload. A simulated computer work composed of mental arithmetic tasks with different levels of difficulty was designed and used in the experiment to measure physiological signals (heart rate, heart rate variability, electromyography, electrodermal activity, and respiration), subjective ratings of mental workload (the NASA Task Load Index), and task performance. The indices from electrodermal activity and respiration had a significant increment as task difficulty increased. There were no significant differences between the average heart rate and the low-frequency/high-frequency ratio among tasks. The classification of mental workload using combined indices as inputs showed that classification models combining physiological signals and task performance can reach satisfying accuracy at 96.4% and an accuracy of 78.3% when only using physiological indices as inputs. The present study also showed that ECG and EDA signals have good discriminating power for mental workload detection. Practitioner summary: The methods used in this study could be applied to office workers, and the findings provide preliminary support and theoretical exploration for follow-up early mental workload detection systems, whose implementation in the real world could beneficially impact worker health and company efficiency. Abbreviations: NASA-TLX: the national aeronautics and space administration-task load index; ECG: electrocardiographic; EDA: electrodermal activity; EEG: electroencephalogram; LDA: linear discriminant analysis; SVM: support vector machine; KNN: k-nearest neighbor; ANNs: artificial neural networks; EMG: electromyography; PPG: photoplethysmography; SD: standard deviation; BMI: body mass index; DSSQ: dundee stress state questionnaire; ANOVA: analysis of variance; SC: skin conductance; RMS: root mean square; AVHR: the average heart rate; HR: heart rate; LF/HF: the ratio between the low frequencies band and the high frequency band; PSD: power spectral density; MF: median frequency; HRV: heart rate variability; BPNN: backpropagation neural network

Comparison of NASA-TLX scale, modified Cooper–Harper scale and mean inter-beat interval as measures of pilot mental workload during simulated flight tasks

The sensitivity of NASA-TLX scale, modified Cooper–Harper (MCH) scale and the mean inter-beat interval (IBI) of successive heart beats, as measures of pilot mental workload (MWL), were evaluated in a flight training device (FTD). Operational F/A-18C pilots flew instrument approaches with varying task loads. Pilots’ performance, subjective MWL ratings and IBI were measured. Based on the pilots’ performance, three performance categories were formed; high-, medium- and low-performance. Values of the subjective rating scales and IBI were compared between categories. It was found that all measures were able to differentiate most task conditions and there was a strong, positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. While NASA-TLX, MCH and IBI have all been previously used to measure MWL, this study is the first one to investigate their association in a modern FTD, using a realistic flying mission and operational pilots.Practitioner summary: NASA-TLX scale, MCH scale and the IBI were evaluated in a flight training device. All measures were able to differentiate most task conditions and there was a positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated.Abbreviations: ANOVA: Analysis of Variance; ECG: Electrocardiograph; F/A: fighter/attack; ft: feet; FTD: flight training device; G: Gravity; km: kilometer; m: meter; m/s: meters per second; MWL: mental workload; MCH: modified cooper-harper; NASA-TLX: NASA Task Load Index; NM: Nautical Mile; NN: normal-to-normal; IBI: inter-beat interval; ILS: Instrument Landing System; RR: R-Wave to R-Wave; SD: standard deviation; TTP: tactics, techniques and procedures; WTSAT: Weapon Tactics and Situation Awareness Trainer

Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation

Mental workload is an important factor during driving, as both high and low levels may result in driver error. This research examined the mental workload of drivers caused by changes in road environment and how such changes impact upon behaviour, physiological responses, eye movements and brain activity. The experiment used functional near infrared spectroscopy to record prefrontal cortex activation associated with changes in mental workload during simulated driving. Increases in subjective ratings of mental workload caused by changes in road type were accompanied by increases in skin conductance, acceleration signatures and horizontal spread of search. Such changes were also associated with increases in the concentration of oxygenated haemoglobin in the prefrontal cortex. Mental workload fluctuates during driving. Such changes can be identified using a range of measures which could be used to inform the development of in-vehicle devices and partially autonomous systems.

Effects of an Automated Conflict Solver on the Visual Activity of Air Traffic Controllers

ERASMUS is a “subliminal” automated aid system designed to reduce air traffic controllers' workload. Prior experiments showed that ERASMUS reduced subjective ratings of mental workload. In this article, the effect of ERASMUS on objective measures of controllers' visual activity was tested in a fully realistic simulation environment. The eye movements of 7 controllers were recorded during experimental traffic sequences, with and without ERASMUS. Consistent with a reduced workload hypothesis, results showed medium to large effects of ERASMUS on the amplitude of saccades, on the time spent gazing at aircraft, and on the distribution of attention over the visual scene.

Cultural influences on the measurement of subjective mental workload

Cognitive ergonomics is well entrenched in North American and most European work environments, where systems and products are designed with the capabilities and limitations of the user in mind. A prominent technique for analysing task demands is subjective mental workload measurement. Subjective ratings of mental workload have been shown—in North American and European populations—to be reliable and sensitive to changes in mental workload. However, there is reason to think that cultural differences may affect subjective ratings. This study compared the performance and subjective mental workload ratings of Indonesian (n = 87) and Dutch (n = 88) students in the context of a hybrid memory/visual search task. Performance was comparable for the two groups, but the sensitivity of the subjective workload measures was better in the Dutch than in the Indonesian group. The results are interpreted in light of social axioms and cultural values. Statement of Relevance: Now more than ever ergonomists must think beyond national borders in improving work processes. Globalisation requires that cultural differences be taken into account in developing and applying ergonomic techniques. This paper presents empirical work showing that cultural differences seriously impact the sensitivity of mental workload measures.

Effect of Overheard Conversations on Bystander Productivity

Overheard cell phone conversations are often perceived as particularly annoying or disruptive. The effect on bystander productivity from overheard cell phone conversations for younger and older adults was investigated for two cognitive tasks, mental arithmetic and proofreading. Performance (accuracy and speeded response) was compared in the presence of overheard cell phone conversations, overheard face-to-face conversation, and in quiet. Subjective ratings of mental workload were also obtained in each condition. In general, overheard cell phone and face-to-face conversations had very similar detrimental effects on performance and were associated with greater ratings of frustration in the arithmetic task. When balanced for number of conversational turns and overall number of words, overheard cell phone conversations do not have a greater effect on bystander productivity than overheard face-to-face conversations.

Time perception as a workload measure in simulated car driving

In experimental studies using flight simulations subjects’ duration estimates have shown to be an effective indicator of cognitive task demands. In this study we wanted to find out whether subjective time perception could serve as a measure of cognitive workload during simulated car driving. Participants drove on a round course of a driving simulator consisting of three different environments with different levels of task demands. Drivers were required to perform a time-production task while driving the vehicle. Electrodermal activity and subjective ratings of mental workload (SWAT) were recorded simultaneously. The length of produced intervals increased significantly in more complex driving situations, as did electrodermal activity and subjective ratings of mental workload. Thus, time production is a valid indicator of cognitive involvement in simulated driving and could become a valid method to measure the current mental workload of car drivers in various traffic situations.

Detecting Mental Workload Fluctuation during Learning of a Novel Task Using Thermography

Training has been identified as a means to reduce mental workload. However, there are limitations associated with many current mental workload measurement techniques. Thermography of the face has been identified as a potential workload measurement tool that may be more closely related to performance and subjective ratings of performance than many other physiological measures. The objective of this study was to validate the efficacy of thermography for assessing mental workload. Twenty participants, 10 males and 10 females, completed seven blocks of an alpha-numeric task. Changes in nose temperature (ANT), task accuracy, reaction time, and two subjective mental workload ratings were collected. Performance improved over time, while ANT and subjective mental workload ratings decreased. Nose temperature was found to be strongly correlated with both performance measures and subjective perceptions of mental workload.

Miles away: determining the extent of secondary task interference on simulated driving

There is a seemingly perennial debate in the literature about the relative merits of using a secondary task as a measure of spare attentional capacity. One of the main drawbacks is that it could adversely affect the primary task, or other measures of mental workload. The present experiment therefore addressed an important methodological issue for the dual-task experimental approach–that of secondary task interference. The current experiment recorded data in both single- and dual-task scenarios to ascertain the level of secondary task interference in the Southampton Driving Simulator. The results indicated that a spatial secondary task did not have a detrimental effect on driving performance, although it consistently inflated subjective mental workload ratings. However, the latter effect was so consistent across all conditions that it was not considered to pose a problem. General issues of experimental design, as well as wider implications of the findings for multiple resources theory, are discussed.

Effects of Imperfect Automation on Decision Making in a Simulated Command and Control Task

Effects of four types of automation support and two levels of automation reliability were examined. The objective was to examine the differential impact of information and decision automation and to investigate the costs of automation unreliability. Research has shown that imperfect automation can lead to differential effects of stages and levels of automation on human performance. Eighteen participants performed a "sensor to shooter" targeting simulation of command and control. Dependent variables included accuracy and response time of target engagement decisions, secondary task performance, and subjective ratings of mental work-load, trust, and self-confidence. Compared with manual performance, reliable automation significantly reduced decision times. Unreliable automation led to greater cost in decision-making accuracy under the higher automation reliability condition for three different forms of decision automation relative to information automation. At low automation reliability, however, there was a cost in performance for both information and decision automation. The results are consistent with a model of human-automation interaction that requires evaluation of the different stages of information processing to which automation support can be applied. If fully reliable decision automation cannot be guaranteed, designers should provide users with information automation support or other tools that allow for inspection and analysis of raw data.

Interactive effects of mental and postural demands on subjective assessment of mental workload and postural stability

Attentional resources are required to maintain an upright posture while completing concurrent mental tasks, with allocation dependent on the nature and complexity of the individual tasks. If available resources are exceeded, either mental or balance performance may be compromised. In this study, the effects of several levels of mental and postural demands were determined using subjective assessment of mental workload and postural stability. Mental activity performed, visual condition, and postural stance were the independent variables. Each of 18 conditions, comprised of a mental task and sustained upright stance, lasted for one minute. Perceived mental workload and postural stability were obtained using a visual analog scale and postural stability scale, respectively. Increases in the difficulty of the standing task did not significantly affect cognitive performance, although larger subjective ratings of mental workload indicated that individuals perceived an increase in difficulty associated with the mental task. Perceptions of postural stability, however, did reflect changes in objective postural sway measures caused by alterations in mental demands. The results overall suggest that a state of heightened arousal, caused by physical/environmental changes or mental demands, may lead to changes in the effectiveness of the visual analog scale, whereas the postural stability scale can still provide a reasonable indication of postural sway.

Context effects in subjective mental workload ratings.

The impact of performance context on subjective mental workload ratings was assessed with the Subjective Workload Assessment Technique (SWAT) and the NASA Task Load Index (TLX). In Experiment 1, a strong context effect was demonstrated. A low range of task difficulty produced considerably higher ratings on a common set of difficulty levels than did a high range of task difficulty. In Experiment 2, increasing the participants' range of experiences during practice eliminated the context effect. We recommend that methods for standardizing context, such as providing experience with the complete difficulty range, be developed for subjective mental workload evaluations. Actual or potential applications of this research include providing methodologies for controlling context effects in practical assessments of mental workload to increase the validity of subjective measures.

Small Fields of view Interfere with Perception of Heading during Active Control But not Passive Viewing

Dyre, Warren, and Garness (1996) found that heading performance during active control differed qualitatively from heading performance during passive viewing, though it was unclear exactly why. Our study examined the relationships between field of view, mental workload, and heading perception. We assessed heading performance during active control and passive viewing with a 30° field of view (Dyre et al. used 90°). Subjective mental workload ratings were collected to assess its relationship to active and passive perception of heading. We found results contrary to those of Dyre et al. (1996): heading perception was less accurate during active control than passive viewing and the relationship between simulated velocity and heading performance was reversed. Further, subjective mental workload was higher during active control than during passive viewing. These results suggest that field of view is an important optical variable mediating heading perception during active control.

The Effect of Delayed Report on Subjective Ratings of Mental Workload

Forty-eight subjects performed a short-term memory task with several difficulty levels and provided either immediate or delayed ratings of workload via the Subjective Workload Assessment Technique (SWAT). Mean SWAT ratings did not vary significantly as a function of delayed report, but a substantial number of subjects gave delayed ratings that were discrepant from their immediate ratings. A counterbalancing effect in delayed ratings appears to have been a factor in the failure of the delay effect to reach significance. A secondary objective of this study was to examine the sensitivity of SWAT in a between-subjects design. SWAT ratings varied significantly as a function of task difficulty manipulations, supporting the sensitivity of SWAT to the workload of the conditions used.