Simulated Flight Task Research Articles

Cardiac autonomic responses in relation to cognitive workload during simulated military flight

Understanding the operator's cognitive workload is crucial for efficiency and safety in human–machine systems. This study investigated how cognitive workload modulates cardiac autonomic regulation during a standardized military simulator flight. Military student pilots completed simulated flight tasks in a Hawk flight simulator. Continuous electrocardiography was recorded to analyze time and frequency domain heart rate variability (HRV). After the simulation, a flight instructor used a standardized method to evaluate student pilot's individual cognitive workload from video-recorded flight simulator data. Results indicated that HRV was able to differentiate flight phases that induced varying levels of cognitive workload; an increasing level of cognitive workload caused significant decreases in many HRV variables, mainly reflecting parasympathetic deactivation of cardiac autonomic regulation. In conclusion, autonomic physiological responses can be used to examine reactions to increased cognitive workload during simulated military flights. HRV could be beneficial in assessing individual responses to cognitive workload and pilot performance during simulator training.

Complexity of the instantaneous frequency variation in auditory steady-state response: A high sensitivity, high anti-interference index of mental fatigue

The quantification of mental fatigue (MF) in aviation remains a long-lasting problem which has a bearing on flight safety. In this study, we have developed a complexity-based MF index based on multiscale entropy (MSE) analysis of instantaneous frequency variation (IFV) in auditory steady-state response (ASSR). Subjects were asked to undertake a flight simulation task, before and after which, electroencephalograms (EEGs) were recorded with no auditory stimulus, and with Don chirp sound to elicit ASSR. In the anti-interference tests, our proposed MF index showed higher performance in terms of anti-interference from electrooculogram (EOG) and environment. In the flight simulation task, fatigue evaluations based on ASSR have achieved a better fatigue detection effect comparing with the conventional methods: 95% in the fatigue detection rate and −13.885% in the MF index. In conclusion, the proposed MF index has high sensitivity and high anti-interference capacities, therefore has potential applications in a variety of fields that require robust monitoring of MF outside a shielded chamber.

Effects of Task Proficiency and Automation Reliability on Pilot-ePilot Teaming in Intelligent Aircraft Cockpits

Intelligence of aircraft cockpits can be improved by creating an electronic pilot (ePilot) to establish a Pilot-ePilot teaming relationship. Effects of task proficiency and automation reliability on such kind of human-autonomy teaming were studied, including task performance, task load, situation awareness, trust in automation, and human errors. In a 2 × 3 within-subjects experiment (low, medium & high levels for task proficiency and automation reliability), 21 male participants completed simulated flight tasks within local airport traffic patterns. Participants were teamed up with a system which took over the flight control priority based on task requirements and cognitive states. Experimental data was analyzed by statistical tests on significance and effect sizes. Results indicated that automation reliability influenced subjective and objective measures more than task proficiency, while task proficiency partly compensated the loss of performance due to deteriorated automation reliability. Findings support that the pilot-centered design principle should still be adhered to for Pilot-ePilot teaming in intelligent aircraft cockpits.

EEG-based mental workload assessment in a simulated flight task using single-channel brain rhythm sequence

EEG-based mental workload assessment in a simulated flight task using single-channel brain rhythm sequence

Neck Muscle Coactivation Response to Varied Levels of Mental Workload During Simulated Flight Tasks.

To evaluate neck muscle coactivation across different levels of mental workload during simulated flight tasks. Neck pain (NP) is highly prevalent among military aviators. Given the complex nature within the flight environment, mental workload may be a risk factor for NP. This may induce higher levels of neck muscle coactivity, which over time may accelerate fatigue, increase neck discomfort, and affect flight task performance. Three counterbalanced mental workload conditions represented by simulated flight tasks modulated by interstimulus frequency and complexity were investigated using the Modifiable Multitasking Environment (ModME). The primary measure was a neck coactivation index to describe the neuromuscular effort of the neck muscles as a system. Additional measures included perceived workload (NASA TLX), subjective discomfort, and task performance. Participants (n = 60; 30M, 30F) performed three test conditions over 1hr each while seated in a simulated seating environment. Neck coactivation indices (CoA) and subjective neck discomfort corresponded with increasing level of mental workload. Average CoAs for low, medium, and high workloads were: .0278(SD = .0232), .0286(SD = .0231), and .0295(SD = .0228), respectively. NASA TLX mental, temporal, effort, and overall scores also increased with the level of mental workload assigned. For ModME task performance, the overall performance score, monitoring accuracy, and resource management accuracy decreased while reaction times increased with the increasing level of mental workload. Communication accuracy was lowest with the low mental workload but had higher reaction times relative to increasing workload. Mental workload affects neck muscle coactivation during combinations of simulated flight tasks within a simulated helicopter seating environment. The results of this study provide insights into the physical response to mental workload. With increasing multisensory modalities within the work environment, these insights may assist the consideration of physical effects from cognitive factors.

Beyond performance: the role of task demand, effort, and individual differences in ab initio pilots

Aviation safety depends on the skill and expertise of pilots to meet the task demands of flying an aircraft in an effective and efficient manner. During flight training, students may respond differently to imposed task demands based on individual differences in capacity, physiological arousal, and effort. To ensure that pilots achieve a common desired level of expertise, training programs should account for individual differences to optimize pilot performance. This study investigates the relationship between task performance and physiological correlates of effort in ab initio pilots. Twenty-four participants conducted a flight simulator task with three difficulty levels and were asked to rate their perceived demand and effort using the NASA TLX. We recorded heart rate, EEG brain activity, and pupil size to assess changes in the participants’ mental and physiological states across different task demands. We found that, despite group-level correlations between performance error and physiological responses, individual differences in physiological responses to task demands reflected different levels of participant effort and task efficiency. These findings suggest that physiological monitoring of student pilots might provide beneficial insights to flight instructors to optimize pilot training at the individual level.

Simultaneous fMRI and tDCS for Enhancing Training of Flight Tasks.

There is a gap in our understanding of how best to apply transcranial direct-current stimulation (tDCS) to enhance learning in complex, realistic, and multifocus tasks such as aviation. Our goal is to assess the effects of tDCS and feedback training on task performance, brain activity, and connectivity using functional magnetic resonance imaging (fMRI). Experienced glider pilots were recruited to perform a one-day, three-run flight-simulator task involving varying difficulty conditions and a secondary auditory task, mimicking real flight requirements. The stimulation group (versus sham) received 1.5 mA high-definition HD-tDCS to the right dorsolateral prefrontal cortex (DLPFC) for 30 min during the training. Whole-brain fMRI was collected before, during, and after stimulation. Active stimulation improved piloting performance both during and post-training, particularly in novice pilots. The fMRI revealed a number of tDCS-induced effects on brain activation, including an increase in the left cerebellum and bilateral basal ganglia for the most difficult conditions, an increase in DLPFC activation and connectivity to the cerebellum during stimulation, and an inhibition in the secondary task-related auditory cortex and Broca's area. Here, we show that stimulation increases activity and connectivity in flight-related brain areas, particularly in novices, and increases the brain's ability to focus on flying and ignore distractors. These findings can guide applied neurostimulation in real pilot training to enhance skill acquisition and can be applied widely in other complex perceptual-motor real-world tasks.

Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement

Mental fatigue (MF) can impair pilots’ performance and reactions to unforeseen events and is therefore an important concept within aviation. The physiological measurement of MF, especially with EEG and, in recent years, fNIRS, has gained much attention. However, a systematic investigation and comparison of the measurements is seldomly done. We induced MF via time on task during a 90-min simulated flight task and collected concurrent EEG-fNIRS, performance and self-report data from 31 participants. While their subjective MF increased linearly, the participants were able to keep their performance stable over the course of the experiment. EEG data showed an early increase and levelling in parietal alpha power and a slower, but steady increase in frontal theta power. No consistent trend could be observed in the fNIRS data. Thus, more research on fNIRS is needed to understand its possibilities and limits for MF assessment, and a combination with EEG is advisable to compare and validate results. Until then, EEG remains the better choice for continuous MF assessment in cockpit applications because of its high sensitivity to a transition from alert to fatigued, even before performance is impaired.

Development of a quantitative measurement on visual clutter in see through display.

With the wide use of transmission displays to improve operation performance, the display information highlights clutter because of the contradiction between the massive amount of information and limited display area. Our study aimed to develop a quantitative measurement for declutter design and appraisal. Using the ergonomics research system of characters and symbols in a see-through cockpit display, we set the simulated flight task interface at four pixel scale levels by enlarging all the display elements in a certain ratio. Flight task videos of 12 clutter degrees were recorded using each flight interface matched with three flight scene complexity levels. A total of 60 pilots completed the visual search tasks in the flight task video while the eye tracker was used to record the view path in real time. Visual search performance was analyzed to study the effect of various clutter factors and levels on pilots' performance in visual search tasks, and acquire quantitative clutter measure parameters. GLM univariate test revealed that there were significant differences among the fixation time in areas of interest (AOI), total Fixation point number, total fixation time at four pixel scale levels, and three flight scene complexity levels (P < 0.05). Visual search performance declined after the cutoff point, while the clutter degree increased. According to the visual search performance data, the recommend feature congestion upper pixel number limit in a 600*800 display was 18,576, and the pixel ratio was 3.87%. A quantitative measurement for declutter design and appraisal of cockpit displays was developed, which can be used to support see-through display design.

Mental fatigue decreases complexity: Evidence from multiscale entropy analysis of instantaneous frequency variation in alpha rhythm.

Mental fatigue (MF) jeopardizes performance and safety through a variety of cognitive impairments and according to the complexity loss theory, should represent "complexity loss" in electroencephalogram (EEG). However, the studies are few and inconsistent concerning the relationship between MF and loss of complexity, probably because of the susceptibility of brain waves to noise. In this study, MF was induced in thirteen male college students by a simulated flight task. Before and at the end of the task, spontaneous EEG and auditory steady-state response (ASSR) were recorded and instantaneous frequency variation (IFV) in alpha rhythm was extracted and analyzed by multiscale entropy (MSE) analysis. The results show that there were significant differences in IFV in alpha rhythm either from spontaneous EEG or from ASSR for all subjects. Therefore, the proposed method can be effective in revealing the complexity loss caused by MF in spontaneous EEG and ASSR, which may serve as a promising analyzing method to mark mild mental impairments.

Effect of video game experience on the simulated flight task: the role of attention and spatial orientation

ABSTRACT Objective In this study, we investigated the advantages of attention allocation and spatial orientation among video game players (VGPs) and non-video game players (NVGPs) and explored the performance differences between the two groups in flight simulations. Method Thirty candidates from a Chinese university were categorised as VGPs and NVGPs before participating in all tests. Results The comparison of flight performance and eye movement indicators between the two groups showed that the flight performance of VGPs was significantly better than that of NVGPs. We then found that an attention shift task and group factors predicted flight performance during take-off, while spatial orientation and tracking tasks have direct and indirect effects on cruise task performance, respectively. Eye movement indicators can directly predict flight performance to a certain extent. Conclusion We believe that the transfer effect of game experience in simulated flight tasks assisted VGPs in using top-down processing strategies in the flight process, and in better allocating cognitive resources. KEY POINTS What is already known about this topic: The advantages of attention allocation and spatial orientation among video game players (VGPs) and non-video game players (NVGPs). This difference not only caused by gaming experience, but also comes from the interaction of gaming experience and individual differences. The VGPs perform better on simulated flight mission. We speculated that gaming practice had assisted VGPs in using top-down processing strategies in the flight process, and in better allocating cognitive resources. VGPs did not depend solely on low-level cognitive abilities to finish tasks, but rather could rapidly understand the relevant situation and task in order to choose a proper attention allocation strategy, that visual search patterns must be adapted according to different situations, as opposed to reliance on a single parallel or sequence search. What this topic adds: The promotional effects of gaming experience occurred in high-level cognitive processing, but this effect was also observed in low-level ability tests. The key to promoting and maintaining excellent long-term performance are comprehensive cognitive skills, advanced cognitive skills, the establishment of mental models, and adaptive top-down processing strategies. The aviation training system design should not stop at elemental training to improve general cognitive abilities, but should be a comprehensive ability-training game in order to improve training efficiency for pilots. In order to establish a cognitive ability video game training system for pilots in the future, further exploration is needed to understand whether the transfer effect would occur in real world flight missions.

Investigation of Pilots' Visual Entropy and Eye Fixations for Simulated Flights Consisted of Multiple Take-Offs and Landings

Eye movement characteristics might provide insights on pilots' mental fatigue during prolonged flight. The visual entropy, eye fixation numbers, and eye fixation durations of ten novice pilots and ten expert pilots were analyzed for a four-hour simulated flight task consisting of four consecutive flight legs. Each flight leg lasted approximately one hour and contained five flight phases: takeoff, climb, cruise, descend, and landing. The pilots maneuvered the simulated B-52 aircraft following instrument flight rules (IFR) in a moderate-fidelity Microsoft Flight Simulator environment. Our results indicate that experts’ eye movement characteristics were significantly different from those of novices. In detail, novices' eye movements were more random, produced longer eye fixation durations, and had fewer eye fixation numbers on the areas of interest (AOIs) than the experts. In addition, the repetitive task (i.e., four consecutive flights) significantly impacted the eye movement characteristics for both experts and novices. Visual entropy and eye fixation duration increased, while eye fixation numbers decreased for both groups as the repetition index increased. Finally, the flight phases also affected eye movement characteristics. The results show that both experts' and novices' visual entropies were relatively higher during climb, cruise, and descend phases, whereas those were relatively lower during the takeoff and landing phases. The present results provide a foundation for us to better understand the similarities and dissimilarities of eye movement characteristics between the experts and novices for a prolonged flight. Lastly, potential scaffolding training methods and pilot anomaly alerting systems, derived from such eye movements, are introduced.

Trust in Automation: The Effects of System Certainty on Decision-Making

In the field of Human Factors, the concept of trust in automation can help to explain how and why users interact with particular systems. One way to examine trust is through task performance and/or behavioral observations. Previous work has identified several system-related moderators of trust in automation, such as reliability and complexity. However, the effects of system certainty, i.e., the knowledge that a machine has regarding its own decision-making abilities, on trust remains unclear. The goal of this study was to examine the extent to which system certainty affects perceived trust. Participants performed a partially simulated flight task and decided what action to take in response to targets in the environment detected by the aircraft’s automation. The automation’s certainty levels in recognizing targets were 30%, 50%, and 80%. Overall, participants accepted the system’s recommendation regardless of the certainty level and trust in the system increased as the system’s certainty level increased. Results may help to inform the development of future autonomous systems.

Effects of Air Route Alternation and Display Design on an Operator’s Situation Awareness, Task Performance and Mental Workload in Simulated Flight Tasks

Air route alternation caused by unexpected events in abnormal or emergency situations often produces adverse consequences on an operator’s cognition and behavior in flight tasks. Under such a circumstance, it is especially necessary to examine the utility of the interaction displays usually designed based on the routine environment. This study was aimed to investigate the effects of air route alternation and display design on operators’ situation awareness (SA), task performance and mental workload during simulated flight tasks. Twenty-four participants attended an experiment where they were instructed to perform simulated flight tasks with three types of display designs in both air-route-as-planned and air-route-altered conditions. Subjective measures, behavioral measures and eye movement measures were adopted to assess the participants’ SA, task performance and mental workload. The results show that unexpected air route alternation increases mental workload as well as deteriorates the SA and task performance due to the gap between attention resource demand and supply. Reducing the demand of the operator’s attention resource should be the focus when coping with unexpected events in abnormal situations. In addition, reasonable information layout, such as a center-layout design of the critical decision-making information, is more important than information salience for improving the SA and task performance in abnormal situations. Nevertheless, indicators with a high-salience design, such as a more open window design and immersive design, are still worth recommending.

Nonholonomic Yaw Control of an Underactuated Flying Robot With Model-Based Reinforcement Learning

Nonholonomic control is a candidate to control nonlinear systems with path-dependant states. We investigate an underactuated flying micro-aerial-vehicle, the ionocraft, that requires nonholonomic control in the yaw-direction for complete attitude control. Deploying an analytical control law involves substantial engineering design and is sensitive to inaccuracy in the system model. With specific assumptions on assembly and system dynamics, we derive a Lie bracket for yaw control of the ionocraft. As a comparison to the significant engineering effort required for an analytic control law, we implement a data-driven model-based reinforcement learning yaw controller in a simulated flight task. We demonstrate that a simple model-based reinforcement learning framework can match the derived Lie bracket control (in yaw rate and chosen actions) in a few minutes of flight data, without a pre-defined dynamics function. This paper shows that learning-based approaches are useful as a tool for synthesis of nonlinear control laws previously only addressable through expert-based design.

Symptoms of convergence and accommodative insufficiency predict engagement and cognitive fatigue during complex task performance with and without automation

Deficits in the accommodative and/or vergence responses have been linked with inattentive behavioral symptoms. While using automated systems (e.g., self-driving cars, autopilot), operators (e.g., drivers, pilots, soldiers) visually monitor displays for critical changes, making deficits in the accommodative and/or vergence responses potentially hazardous for individuals remaining actively engaged in the task at hand. The purpose of this study was to determine if symptoms of accommodative-vergence deficits predict an individual's level of task engagement and cognitive fatigue while performing a flight simulation task with or without automation. Eighty-four participants performed a flight simulation task with or without automation. Prior to task completion, self-report accommodative-convergence deficit symptoms were assessed with the Convergence Insufficiency Symptom Survey (CISS). Before and after the flight simulation task participants rated their task engagement and cognitive fatigue. Electroencephalographic activity (EEG) was recorded concurrently during task performance. Results showed that higher scores on the CISS were related to increased feelings of fatigue and decreased ratings of task engagement. The CISS was also positively related to parietal-occipital fast alpha power during the last 10 min of the task for participants using automation, suggesting increased cortical idling. CISS scores did not predict performance. Results have implications for optimizing operator cognitive states over extended task performance.

Exploring Residual Capacity

Abstract. Alarm systems may take advantage of the tactile modality for allocation of attentional resources during the performance of demanding tasks in complex environments. The purpose of this study was to determine the effectiveness of a tactile warning during increasing levels of mental workload in a primary task. Three simulated flight task conditions varying in mental workload were presented while an “on-thigh” vibrotactile warning was randomly assessed. Generally, there was a decrement in overall warning response performance when task workload increased, but this tendency faded and plateaued as the level of task workload progressed. The observed pattern indicates that vibrotactile warning signals may offer a plausible mode for conveying information during increasing levels of primary task workload.

Efficacy of Dry Electroencephalography and Cognitive Assessment in Identifying Normobaric Hypoxia in Designated Aviators

The purpose of this study was to investigate the use of EEG and cognitive functioning assessments to identify hypoxia in naval aviators. Naval aviators were brought slowly to a 6,096 m exposure to induce normobaric hypoxia while completing a cognitive assessment and a simulated flight task. There were significant reductions in cognitive performance at 6,096 m when compared to sea level. There were no significant changes in EEG-recorded brainwave activity during the 6,096 m exposure compared to sea level. There were also no significant decrements in flight performance at 6,096 m. Additional research is needed to better understand the impact of hypoxia on brainwave activity and pilot performance.

Utility of Concurrent Bandwidth Feedback in Training Aircraft Flight Tasks

The effects of concurrent bandwidth feedback on operator performance and workload was analyzed in training an aircraft flight control task. In the experiment, participants completed a simulated flight task consisting of three complexity levels using traditional flight instruments. Thirty participants were divided into equal sized control and feedback groups. The control group controlled simulated aircraft motion with visual guidance for pitch, roll, and altitude provided by traditional flight instruments. The feedback group received additional visual concurrent bandwidth feedback for each controlled degree of freedom. For both groups, performance and workload measurements were evaluated to determine the effects of the feedback on subject learning rate and maximum skill level. To assess short-term retention of learned skill for the feedback group, the concurrent feedback was removed, and performance was again evaluated. Statistical analyses showed that participants in the feedback group immediately performed better than those in the control group, that the performance difference between the two groups was more pronounced for more complex tasks, and that final performance levels for the feedback group significantly exceeded that of the control group. We found that concurrent bandwidth feedback does not reduce workload in our flight tasks, and that for the short periods tested, participants continued to perform at the same performance and workload levels when the feedback was removed.

Differentiating Active and Passive Fatigue with the use of Electroencephalography

Desmond and Hancock (2001) proposed that two types of cognitive fatigue can arise depending on the nature of a task: active or passive. The purpose of this study was to use electroencephalographic indices of task engagement and a candidate marker of strain to differentiate these two fatigue states. Participants ( N = 84) performed a generalized flight simulation task for 62 min under either active, passive, or control conditions. Passive fatigue was characterized by reduced EEG engagement and initially elevated and stable ratios of Fz theta to POz alpha power compared to active fatigue. No performance differences were found between the fatigue conditions; however, an overall speed-accuracy trade-off was observed from pre to post fatigue induction. These results support theoretical and empirical distinctions between active and passive fatigue and have potential applications for developing augmented cognition technologies to deliver appropriate fatigue countermeasures in automated operational environments.