Variations In Mental Workload Research Articles

Using pupillometry and gaze-based metrics for understanding drivers’ mental workload during automated driving

This Horizon2020-funded driving simulator-based study on automated driving investigated the effect of different car-following scenarios, and takeover situations, on drivers’ mental workload, as measured by eye tracking-based metrics of pupil diameter and self-reported workload ratings. This study incorporated a mixed design format, with 16 drivers recruited for the SAE Level 2 (L2; SAE International, 2021) automation group, who were asked to monitor the driving and road environment during automation, and 16 drivers in the Level 3 (L3) automation group, who engaged in a non-driving related task (NDRT; Arrows task) during automation. Drivers in each group undertook two experimental drives, lasting about 18 min each. To manipulate perceived workload, difficulty of the driving task was controlled by incorporating a lead vehicle which maintained either a Short (0.5 s) or Long (1.5 s) Time Headway (THW) condition during automated car-following (ACF). Each ACF session was followed by a subsequent request to takeover, which happened either in the presence or absence of a lead vehicle. Results from standard deviation of pupil diameter values indicated that drivers’ mental workload levels fluctuated significantly more when monitoring the drive during L2 ACF, compared to manual car-following (MCF). Additionally, we found that drivers’ mental workload, as indicated by their mean pupil diameter, increased steeply around takeovers, and was further exacerbated by the presence of a lead vehicle during the takeovers, especially in the Short THW condition, for both groups. Pupil diameter was found to be sensitive to subtle variations in mental workload, and closely resembled the trend seen in self-reported workload ratings. Further research is warranted to assess the feasibility of using eye-tracking-based metrics along with other physiological sensors, especially in real-world settings, to understand whether they can be used as real-time indicators of drivers’ mental workload, in future driver state monitoring systems.

Use of Eye-Tracking to Detect Variations in Mental Workload While Learning to Operate a Physically Coupled Robot

Use of Eye-Tracking to Detect Variations in Mental Workload While Learning to Operate a Physically Coupled Robot

Sensitivity of Eye-Tracking Measures to Variations in Mental Workload while Learning to Operate a Physically Coupled Robot

Sensitivity of Eye-Tracking Measures to Variations in Mental Workload while Learning to Operate a Physically Coupled Robot

Sensitivity of the Spatial Distribution of Fixations to Variations in the Type of Task Demand and Its Relation to Visual Entropy.

Ocular activity is known to be sensitive to variations in mental workload, and recent studies have successfully related the distribution of eye fixations to the mental load. This study aimed to verify the effectiveness of the spatial distribution of fixations as a measure of mental workload and its sensitivity to different types of demands imposed by the task: mental, temporal, and physical. To test the research hypothesis, two experimental studies were run: Experiment 1 evaluated the sensitivity of an index of spatial distribution (Nearest Neighbor Index; NNI) to changes in workload. A sample of 30 participants participated in a within-subject design with different types of task demands (mental, temporal, physical) applied to Tetris game; Experiment 2 investigated the accuracy of the index through the analysis of 1-min epochs during the execution of a visual-spatial task (the “spot the differences” puzzle game). Additionally, NNI was compared to a better-known ocular mental workload index, the entropy rate. The data analysis showed a relation between the NNI and the different workload levels imposed by the tasks. In particular: Experiment 1 demonstrated that increased difficulty, due to higher temporal demand, led to a more dispersed pattern with respect to the baseline, whereas the mental demand led to a more grouped pattern of fixations with respect to the baseline; Experiment 2 indicated that the entropy rate and the NNI show a similar pattern over time, indicating high mental workload after the first minute of activity. That suggests that NNI highlights the greater presence of fixation groups and, accordingly, the entropy indicates a more regular and orderly scanpath. Both indices are sensitive to changes in workload and they seem to anticipate the drop in performance. However, the entropy rate is limited by the use of the areas of interest, making it impossible to apply it in dynamic contexts. Conversely, NNI works with the entire scanpath and it shows sensitivity to different types of task demands. These results confirm the NNI as a measure applicable to different contexts and its potential use as a trigger in adaptive systems implemented in high-risk settings, such as control rooms and transportation systems.

Combined assessment of attentional reserve and cognitive-motor effort under various levels of challenge with a dry EEG system.

A novel ERP approach was proposed to index variations in mental workload, particularly in attentional reserve, which is complementary to EEG spectral content thought to reflect mental effort. To our knowledge, no study has assessed mental effort and attentional reserve simultaneously in EEG gel-based and, importantly, dry systems, which are particularly well suited for real-world settings. Therefore, by systematically considering ERP, EEG spectral, and importantly the combination of both, this study examined if a small set of dry EEG electrodes could detect changes in both spectral and ERP metrics to assess the mental workload under various challenges with a similar fidelity to their gel-based counterparts in a laboratory setting. By employing both EEG gel-based and dry systems, the ERP and spectral markers were computed while participants executed a visuomotor task under three levels of challenge. For both EEG systems, more challenging levels of difficulty were associated with concomitant changes in ERP amplitude, and spectral power reflected a reduction of the attentional reserve and an increase in cognitive-motor effort, respectively. Those variations in attentional reserve and cognitive-motor effort collectively indexed mental workload with nearly identical fidelity for both gel-based and dry EEG systems. These findings promise to assess the mental workload in situations where the use of dry EEG systems could be advantageously employed to examine human cognitive-motor performance.

Spatial Pattern of Eye Fixations and Evidence of Ultradian Rhythms in Aircraft Pilots.

Eye fixations can be distributed in three ways: randomly, in clusters, and regularly. However, there is always a continuum among these types, because these spatial patterns are the result of a process evolving over time. The focus of the present work was to study the changes over time observed in the Nearest Neighbor Index (NNI), an index derived from the spatial distribution of eye fixations that has been reported to be sensitive to variations in mental workload. Of particular interest are periodic changes in the ultradian timescale (an ultradian rhythm is a recurrent period or cycle repeated throughout a 24-h circadian day). Data from a previously reported experiment were further analyzed using temporal spectral analysis, which is one of the most commonly used techniques for studying measurements collected at regularly spaced intervals of time. An ultradian rhythm with a periodicity between 2 and 15 min was found, which is compatible with results obtained by analyzing reaction times in prolonged vigilance tasks. The identification of a periodicity in the allocation of mental resources should be considered in the design of automation support that is dynamically matched to mental workload.

Identification of temporal variations in mental workload using locally-linear-embedding-based EEG feature reduction and support-vector-machine-based clustering and classification techniques

Identifying the abnormal changes of mental workload (MWL) over time is quite crucial for preventing the accidents due to cognitive overload and inattention of human operators in safety-critical human–machine systems. It is known that various neuroimaging technologies can be used to identify the MWL variations. In order to classify MWL into a few discrete levels using representative MWL indicators and small-sized training samples, a novel EEG-based approach by combining locally linear embedding (LLE), support vector clustering (SVC) and support vector data description (SVDD) techniques is proposed and evaluated by using the experimentally measured data. The MWL indicators from different cortical regions are first elicited by using the LLE technique. Then, the SVC approach is used to find the clusters of these MWL indicators and thereby to detect MWL variations. It is shown that the clusters can be interpreted as the binary class MWL. Furthermore, a trained binary SVDD classifier is shown to be capable of detecting slight variations of those indicators. By combining the two schemes, a SVC–SVDD framework is proposed, where the clear-cut (smaller) cluster is detected by SVC first and then a subsequent SVDD model is utilized to divide the overlapped (larger) cluster into two classes. Finally, three-class MWL levels (low, normal and high) can be identified automatically. The experimental data analysis results are compared with those of several existing methods. It has been demonstrated that the proposed framework can lead to acceptable computational accuracy and has the advantages of both unsupervised and supervised training strategies.

The Effects of Driving Environment on the Mental Workload of Train Drivers

A study is carried out to investigate the effects of driving environment on the mental workload of train drivers while driving. The driving task is performed under three environmental conditions, i.e. clear sunny day, rainy day and rainy night driving. Electroencephalography (EEG) measurements are recorded from the Fz and Pz channels of fifteen male subjects aged between 24 to 48 years old. The mean alpha power is monitored as a function of time as this signal reflects the variations in mental workload of the drivers. The results exhibit that the signal pattern for rainy night driving condition is significantly different compared to others. This finding indicates that the train drivers show an increase in mental workload after six minutes of driving under rainy night condition. The results demonstrate a percentage difference in mean alpha power of 37% between daytime and rainy night driving conditions during the early periods of driving. This indicates that the mental workload of train drivers tends to be low with an increased level of sleepiness under such conditions, which are signs of low vigilance.

Monitoring drivers’ mental workload in driving simulators using physiological measures

Many traffic accidents are caused by, or at least related to, inadequate mental workload, when it is either too low (vigilance) or too high (stress). Creating variations in mental workload and accident-prone driving for research purposes is difficult in the real world. In driving simulators the measurement of driver mental workload is relatively easily conducted by means of physiological measures, although good research skills are required and it is time-consuming. The fact that modern driving simulator environments are laboratory-equivalent nowadays allows full control with respect to environmental conditions, scenarios and stimuli, and enables physiological measurement of parameters of mental workload such as heart rate and brain activity. Several examples are presented to illustrate the potential of modern high-standard driving simulator environments regarding the monitoring of drivers’ mental workload during task performance.

Saccadic Peak Velocity Sensitivity to Variations in Mental Workload

For research and applications in the field of (neuro)ergonomics, it is of increasing importance to have reliable methods for measuring mental workload. In the present study we examined the hypothesis that saccadic eye movements can be used for an online assessment of mental workload. Saccadic main sequence (amplitude, duration and peak velocity) was used as a diagnostic measure of mental workload in a virtual driving task with three complexity levels. We tested 18 drivers in the SIRCA driving simulator while their eye movements were recorded. The Wickens' multiple resources model was used as theoretical framework. Changes in mental workload between the complexity levels were evaluated multidimensionally, using subjective rating, performance in a secondary task, and other behavioral indices. Saccadic peak velocity decreased (7.2 visual degrees/s) as the mental workload increased, as measured by scores of mental workload test (15.2 scores) and the increase of the reaction time on the secondary task (46 ms). Saccadic peak velocity is affected by variations in mental workload during ecologically valid tasks. We conclude that saccadic peak velocity could be a useful diagnostic index for the assessment of operators' mental workload and attentional state in hazardous environments.

Approximation of on-line mental workload index in ATC simulated multitasks

To assess the effects of workload pressures, participants interacted with a modified version of air traffic control simulated tasks requiring different levels of cognitive resources. Changes in mental workload between the levels were evaluated multidimensionally using a subjective rating, performance in a secondary task, and other behavioural indices. Saccadic movements were measured using a video-based eye tracking system. The Wickens multiple resource model is used as a theoretical reference framework. Saccadic peak velocity decreases with increasing cognitive load, in agreement with subjective test scores and performance data. That saccadic peak velocity is sensitive to variations in mental workload during ecologically valid tasks is demonstrated.

ASTEF: A simple tool for examining fixations

In human factors and ergonomics research, the analysis of eye movements has gained popularity as a method for obtaining information concerning the operator's cognitive strategies and for drawing inferences about the cognitive state of an individual. For example, recent studies have shown that the distribution of eye fixations is sensitive to variations in mental workload---dispersed when workload is high, and clustered when workload is low. Spatial statistics algorithms can be used to obtain information about the type of distribution and can be applied over fixations recorded during small epochs of time to assess online changes in the level of mental load experienced by the individuals. In order to ease the computation of the statistical index and to encourage research on the spatial properties of visual scanning, A Simple Tool for Examining Fixations has been developed. The software application implements functions for fixation visualization, management, and analysis, and includes a tool for fixation identification from raw gaze point data. Updated information can be obtained online at www.astef.info, where the installation package is freely downloadable.

A Random Glance at the Flight Deck: Pilots' Scanning Strategies and the Real-Time Assessment of Mental Workload

Based on previous research showing the usefulness of spatial statistics in detecting randomness in the distribution of eye fixations, this study investigated the ocular behavior of professional pilots engaged in a simulated flight. The distribution of eye fixations was used as an indirect index of mental workload: Eye movements were recorded during the different phases (departure to landing) of a simulated flight and were analyzed using spatial statistics algorithms. Results showed sensitivity of spatial dispersion indices to variations in mental workload: higher during departure and landing, lower during climb and descend, and the lowest during the cruise phase. This finding provides additional evidence of the utility of fixations distribution as a real-time measure of mental workload and, consequently, as a trigger for adaptive automation.

Using the Distribution of Eye Fixations to Assess Pilots' Mental Workload

Based on previous research showing the usefulness of spatial statistics in detecting randomness in the distribution of eye fixations, this study investigated the ocular behavior of professional pilots engaged in a simulated flight. The distribution of eye fixations is here used an index of mental workload. Eye movements were recorded during the different phases (departure to landing) of a simulated flight, and were analyzed using spatial statistics algorithms. Results showed sensitivity of spatial dispersion indices to variations in mental workload: higher during departure and landing, lower during the other phases. This result provides additional evidence of the utility of fixations distribution as a real-time measure of mental workload and, consequently, as a trigger for adaptive automation.

Workload variation, intrinsic risk and utility in a simulated air traffic control task: Evidence for compensatory effects

Air traffic control operations provide a potential example of risk homeostasis theory (RHT) in a non-transportational domain. This paper addresses the issue of whether reductions in intrinsic risk can be negated by behavioural adjustments of operators. Following variations in mental workload, previous studies have suggested that air traffic controllers appear to adapt to changing demands through more active or more effective strategic control of performance. This effortful compensatory control of performance has the consequence of eliminating what would otherwise be concomitant variations in accident loss. However, most of the empirical support for this proposition is taken from observational studies in which the relative occurrence of different levels of workload is not controlled. Using a simulated air traffic control task, a study is reported here in which conditions of low, moderate and high levels of mental workload are presented in equal duration to participants. The findings indicate that number of errors differed significantly across these conditions of mental workload which is contrary to what RHT would predict. Evidence is found for a general adaptation (or strategy switching) in response to different levels of workload. This supports earlier work in indicating that behavioural adjustments may not be of sufficient magnitude to restore previously existing levels of safety. However, on a notional measure of extrinsic compensation, participants did respond differently according to workload, thus lending limited support to RHT's prediction that utility and intrinsic risk should act in a multiplicative way to form a statistical interaction.

Assessment of mental workload with task-irrelevant auditory probes

A study was performed to examine the utility of an ERP-based irrelevant probe technique for the assessment of variations in mental workload. Ten highly trained Navy radar operators performed a simulated radar-monitoring task which varied in the density and type of targets to be detected and identified. This task was performed in the presence of a series of irrelevant auditory probes which the radar operators were instructed to ignore. Prior to performing the radar-monitoring task the subjects performed a block of auditory detection trials in which they were asked to respond to the occurrence of one of two low probability tones and ignore the other low probability tone along with a higher probability standard tone. ERPs were recorded from the occurrence of the tones in both the baseline and low and high workload radar-monitoring conditions. The amplitude of the N100, N200, and early and late mismatch negativity (MMN) components decreased from the baseline to the low load radar-monitoring task and again with an increase in the difficulty of the radar-monitoring task. P300 amplitude was sensitive only to the introduction of the radar-monitoring task. These results are interpreted with respect to the phenomenon of attentional capture and suggest that the ERP-based irrelevant-probe technique might prove an effective method for the nonintrusive evaluation of increases in mental workload in complex tasks.