Novice Pilots Research Articles

Computational analysis and designing of flight controllers to improve the drone performance for a novice pilot

Computational analysis and designing of flight controllers to improve the drone performance for a novice pilot

The Impact of Various Cockpit Display Interfaces on Novice Pilots' Mental Workload and Situational Awareness: A Comparative Study.

Future airspace is expected to become more congested with additional in-service cargo and commercial flights. Pilots will face additional burdens in such an environment, given the increasing number of factors that they must simultaneously consider while completing their work activities. Therefore, care and attention must be paid to the mental workload (MWL) experienced by operating pilots. If left unaddressed, a state of mental overload could affect the pilot's ability to complete his or her work activities in a safe and correct manner. This study examines the impact of two different cockpit display interfaces (CDIs), the Steam Gauge panel and the G1000 Glass panel, on novice pilots' MWL and situational awareness (SA) in a flight simulator-based setting. A combination of objective (EEG and HRV) and subjective (NASA-TLX) assessments is used to assess novice pilots' cognitive states during this study. Our results indicate that the gauge design of the CDI affects novice pilots' SA and MWL, with the G1000 Glass panel being more effective in reducing the MWL and improving SA compared with the Steam Gauge panel. The results of this study have implications for the design of future flight deck interfaces and the training of future pilots.

Does fiducial marker visibility impact task performance and information processing in novice and low-time pilots?

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behavior, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous.

How flight experience impacts pilots’ decision-making and visual scanning pattern in low-visibility approaches: preliminary evidence from eye tracking

The visual approach is the most accident-prone phase of a flight, especially in low-visibility conditions. This preliminary study aimed to examine the effects of flight experience on pilots’ decision-making and visual scanning pattern in low-visibility approaches. Twenty pilots were separated into two groups based on their flight experience and completed the high- and low-visibility approaches in balanced order using a high-fidelity flight simulator. Pilots’ mental workload and visual scanning patterns were recorded via an eye tracker. The results showed that, compared to less flight-experienced pilots (20%, 3/15), experienced pilots (80%, 4/5) were more likely to make go-around decisions in the low-visibility approaches. Furthermore, they exhibited a more flexible and adaptable visual scanning pattern by quickly shifting their attention, as evidenced by decreased fixations and increased saccades. These findings suggest that the integration of visual scanning strategy and training solution with a marginally meteorological approach may enhance decision-making safety for novice pilots.

The interconnection between evaluated and self-assessed performance in full flight simulator training

This study explores potential disparities between flight instructor evaluations and pilot self-assessments in the context of full flight simulator training. Evaluated performance was based on the Competency-based Training and Assessment framework, a recent development of competency-based education within aviation. Self-assessed performance is derived from survey responses and debriefing interviews. The simulator session involves eight multi-crew pilot training graduates and eight experienced flight captains, encompassing two tasks featuring sudden technical malfucntions during flight. The flight instructor’s evaluations reveal no significant differences in pilot performance. However, disparities become apparent when pilots engaged in reflecting their performance. Novice pilots, despite perceiving both tasks as easy, exhibited an overconfidence that led them to underestimate the inherent risks. Conversely, experienced pilots demonstrated greater caution towards the risks and engaged in discussing possible hazards. Furthermore, this study highlights the challenge of designing flight simulator training that incorporates surprise elements. Pilots tend to anticipate anomalies more readily in simulator training than during actual flights. Thus, this study underscores the importance of examining how pilots reflect on their performance, complementing the assessment of observable indicators and predefined competencies.

Reinforcement learning for shared autonomy drone landings

Novice pilots find it difficult to operate and land unmanned aerial vehicles (UAVs), due to the complex UAV dynamics, challenges in depth perception, lack of expertise with the control interface and additional disturbances from the ground effect. Therefore we propose a shared autonomy approach to assist pilots in safely landing a UAV under conditions where depth perception is difficult and safe landing zones are limited. Our approach is comprised of two modules: a perception module that encodes information onto a compressed latent representation using two RGB-D cameras and a policy module that is trained with the reinforcement learning algorithm TD3 to discern the pilot’s intent and to provide control inputs that augment the user’s input to safely land the UAV. The policy module is trained in simulation using a population of simulated users. Simulated users are sampled from a parametric model with four parameters, which model a pilot’s tendency to conform to the assistant, proficiency, aggressiveness and speed. We conduct a user study (n=28) where human participants were tasked with landing a physical UAV on one of several platforms under challenging viewing conditions. The assistant, trained with only simulated user data, improved task success rate from 51.4 to 98.2% despite being unaware of the human participants’ goal or the structure of the environment a priori. With the proposed assistant, regardless of prior piloting experience, participants performed with a proficiency greater than the most experienced unassisted participants.

Initial Student Attention-Allocation and Flight-Performance Improvements Based on Eye-Movement Data

At the onset of their flight careers, novice pilots often lack clarity regarding the standard attention-allocation pattern. Therefore, to enhance the efficiency of initial flight training, it is crucial for students to develop a comprehensive understanding of flight control and attention-allocation behavior during the learning process. In this study, flight-performance data and eye-movement data from experienced instructors in no-power stall scenarios were collected to create an attention-allocation training course. An experimental group underwent the attention-allocation training course, while a control group followed the traditional teaching curriculum. The disparities between the flight performance and eye-movement indices of the two groups after they completed their respective courses were compared to evaluate the effectiveness of the training. The finding indicate significant differences between the speed losses, altitude losses, and mean course deviations of the instructors and the control group; these indicators had p-values of 0.01, 0.004, and 0.001, respectively. Moreover, significant differences were observed between the altitude losses and mean course deviations of the instructors and the experimental group; these indicators had p-values of 0.006 and 0.001, respectively. The experimental group, which underwent attention-allocation training, exhibited eye-movement indices that closely resembled those of the instructor group, and its instrument scanning was more strategic, thereby resulting in improved flight performance from that of the control group. Additionally, correlations were observed between flight-performance indices and eye-movement indices of the students. Overall, this study demonstrates the effectiveness of an attention-allocation training course designed specifically for a no-power stall scenario. It effectively enhanced the training outcomes of novice pilots, promoted an appropriate allocation of attention to instrument displays, introduced a novel approach to flight training, and ultimately contributed to aviation safety.

Simultaneous fMRI and tDCS for Enhancing Training of Flight Tasks.

There is a gap in our understanding of how best to apply transcranial direct-current stimulation (tDCS) to enhance learning in complex, realistic, and multifocus tasks such as aviation. Our goal is to assess the effects of tDCS and feedback training on task performance, brain activity, and connectivity using functional magnetic resonance imaging (fMRI). Experienced glider pilots were recruited to perform a one-day, three-run flight-simulator task involving varying difficulty conditions and a secondary auditory task, mimicking real flight requirements. The stimulation group (versus sham) received 1.5 mA high-definition HD-tDCS to the right dorsolateral prefrontal cortex (DLPFC) for 30 min during the training. Whole-brain fMRI was collected before, during, and after stimulation. Active stimulation improved piloting performance both during and post-training, particularly in novice pilots. The fMRI revealed a number of tDCS-induced effects on brain activation, including an increase in the left cerebellum and bilateral basal ganglia for the most difficult conditions, an increase in DLPFC activation and connectivity to the cerebellum during stimulation, and an inhibition in the secondary task-related auditory cortex and Broca's area. Here, we show that stimulation increases activity and connectivity in flight-related brain areas, particularly in novices, and increases the brain's ability to focus on flying and ignore distractors. These findings can guide applied neurostimulation in real pilot training to enhance skill acquisition and can be applied widely in other complex perceptual-motor real-world tasks.

Evaluation of Augmented Reality Interactive Print for General Aviation Weather Training

Current aviation weather training for novice pilots provides limited opportunities to practice correlating information in weather-related situations to develop cognitive skills. Current training materials are largely print based: restricted to text and two-dimensional images. However, interactive print training materials can include interactive augmented reality (AR) weather activities overlaid on text and images. This study compared learner-centered interactive print to print-only training. The participants completed both knowledge-based and scenario-based learning activities. The interactive print training activities significantly improved motivation to learn and engagement with the activities. There was no difference between the two groups when rating the perceived effectiveness. Both training groups improved factual knowledge; however, there was no difference between the two training groups. The participants in the interactive print training group performed significantly better in the knowledge-based activities but worse in the scenario-based activities as compared to the print training group. In the AR visual environment, participants may have relied on visual cues rather than the knowledge learned in the text. The evaluation suggests that AR-enhanced training can improve factual knowledge, positively impact motivation and engagement, and provide learning opportunities in a safe environment.

All's well that ends well? Outcome bias in pilots during instrument flight rules

AbstractThe objective of the present study was to examine if the Outcome Bias also occurs in pilots flying under instrument flight rules (IFR). In a scenario‐based survey, 60 pilots evaluated weather‐related decisions made by hypothetical pilots. Participants rated the decisions as better, less risky, and regarded the probability that they would have made the same decision as higher when they were followed by a positive outcome, than when they were followed by a negative outcome. This effect showed likewise for novice pilots and for experienced pilots. These findings could be relevant for the flight‐related decision‐making of pilots, which sometimes is affected by the decisions made by third‐party pilots. In particular, decisions made by other pilots that have led to positive outcomes might be hastily followed, whereas those that have led to negative outcomes might be hastily rejected.

Influencing factors of novice pilot SA based on DEMATEL-AISM method: From pilots’ view

Pilot situation awareness (SA) regulates flight safety, and inexperience may impair novice pilot reliability in SA. This study aims to determine the key influencing factors of novice pilot SA and to analyze the interrelationship and interaction mechanism of the factors. We investigated 55 novice pilots trained at aviation schools and identified the influencing factor index system by the Delphi survey. The method of Decision Making Trial and Evaluation (DEMATEL) combined with Adversarial Interpretive Structure Modeling (AISM) was adopted. The results show that: (1) The influencing factor index system includes 18 factors, divided into four categories: individual factors, team factors, task and human-machine system factors, and cockpit environment factors. (2) Team communication, team cooperation, basic cognitive ability, interface design, occupational age and experience, and authority gradient are the six crucial influencing factors. The former three have the greatest association with other factors, while the latter three are most likely to affect other factors. (3) Team communication, basic cognitive ability, and interface design are root-cause factors, of which team communication is the most fundamental. (4) The results of DEMATEL and AISM are consistent, both disclosing team communication as the fundamental factor with the highest priority, and cockpit environmental factors as the direct influencing factors but most susceptible to other factors. The present study can be viewed as a conducive attempt to extract vital influencing factors of novice pilot SA, and to provide ergonomic insights for determining the priorities to improve novice pilot SA in training and aircraft design for flight safety.

Real-time Pilot Crew's Mental Workload and Arousal Assessment During Simulated Flights for Training Evaluation: a Case Study.

Training assessment is usually done by evaluating information derived from instructor's supervision related to the pilot's operational performance and behavior. However, this approach lacks objective measures, especially regarding the pilots' mental states while accomplishing the flight training tasks. The study therefore aimed at developing and testing a method for gathering and analyzing in real-time pilots' brain activity and skin conductance to improve the training evaluation. In this regard, Novice pilots' neurophysiological signals were acquired throughout multi-crew training sessions. The results demonstrated how the methodology proposed was able to endow real-time pilots' mental workload and arousal assessment for i) better evaluating training progress and operational behavior during the training session, and ii) for objectively comparing different training sessions.

Estimation of Redox Status in Military Pilots during Hypoxic Flight-Simulation Conditions-A Pilot Study.

At high altitude conditions, the low-pressure atmospheric oxygen reduces the generation of energy, thus inducing a decrease in oxygen availability. As a result, endurance flights evoke imbalance in redox signaling, posing a safety risk for the pilots involved. The aim of the present study was to assess changes in the redox status of military pilots during flight simulation conditions according to their flight hours (experts vs. novice). A total of seven expert pilots and an equal number of novice pilots (trainees) were recruited from the Center for Airforce Medicine of the Greek Military Airforce. Glutathione (GSH) levels, catalase activity (CAT), total antioxidant capacity (TAC), lipid peroxidation through the thiobarbituric acid-reactive substances (TBARS), and protein oxidative damage through the assay of protein carbonyls (PCs) levels were assessed at two time points, once prior to and once immediately post a scheduled flight simulation. In the experienced pilots’ arms, GSH was significantly increased post-flight simulation, with TAC being simultaneously reduced. On the other hand, in the trainees’ arms, CAT and TAC were both increased post-flight. No differences were noted with regard to the TBARS and PCs post-simulation. When the two groups were compared, TAC and PCs were significantly lower in the trainees compared to the experienced pilots. The present study provides useful insight into the physiological redox status adaptations to hypobaric hypoxic flight conditions among pilots. In a further detail, an increase in GSH response post-flight simulation is being evoked in more experienced pilots, indicating an adaptation to the extreme flight conditions, as they battle oxidative stress.

FlyBrate: Evaluating Vibrotactile Cues for Simulated Flight

Contemporary aircraft cockpits rely mostly on audiovisual information propagation which can overwhelm particularly novice pilots. The introduction of tactile feedback, as a less taxed modality, can improve the usability in this case. As part of a within-subject simulator study, 22 participants are asked to fly a visual-flight-rule scenario along a predefined route and identify objects in the outside world that serve as waypoints. Participants fly two similar scenarios with and without a tactile belt that indicates the route. Results show that with the belt, participants perform better in identifying objects, have higher usability and user experience ratings, and a lower perceived cognitive workload, while showing no improvement in spatial awareness. Moreover, 86% of the participants state that they prefer flying with the tactile belt. These results suggest that a tactile belt provides pilots with an unobtrusive mode of assistance for tasks that require orientation using cues from the outside world.

Postural Control and Psychophysical State Following of Flight Simulator Session in Novice Pilots.

Flight simulators can cause side effects usually called simulator sickness. Scientific research proves that postural instability can be an indicator of the occurrence of simulator sickness symptoms. This study aims to assess changes of postural control and psychophysical state in novice pilots following 2-h exposure to simulator conditions. The postural sway was quantified based on variables describing the displacement of the Center of Pressure (COP) generated in a quiet stance with eyes open (EO) and closed (EC). The psychophysical state was assessed using the Simulator Sickness Questionnaire (SSQ). The research was carried out in a group of 24 novice pilots who performed procedural and emergency flight exercises in the simulator at Instrument Meteorological Conditions. Each subject was examined twice: immediately before the simulator session (pre-exposure test), and just after the session (post-exposure test). The differences in postural stability between pre- and post-exposure to simulator conditions were assessed based on the normalized Romberg quotients, calculated for individual variables. The lower median values of all Romberg quotients confirmed the decreasing difference between the measures with eyes open and with eyes closed in the post-exposure tests. After the flight simulator session in both measurements (EO and EC) the values of the length of sway path (SP), the mean amplitude (MA), the sway area (SA) have changed. The visual contribution to postural sway control was reduced. The median values for all SSQ scores (total, nausea, oculomotor, and disorientation scales) were significantly higher in post-exposure tests. The largest increase was noted in the oculomotor SSQ scores (from 7.6 ± 7.6 to 37.9 ± 26.5). Over 50% of pilots participating in this study expressed symptoms typical of simulator sickness connected with visual induction: fatigue, eyestrain, difficulty focusing and difficulty concentrating. The severity of oculomotor and disorientation symptoms were rated as moderate (total SSQ score of more than 25 and <60). This study concludes that changes noted in the postural control and psychophysical state of the studied pilots after exposure to the flight simulator confirm the occurrence of the simulator sickness symptoms. Although, we did not find significant correlation of postural stability with SSQ scores.

Investigation of Pilots' Visual Entropy and Eye Fixations for Simulated Flights Consisted of Multiple Take-Offs and Landings

Eye movement characteristics might provide insights on pilots' mental fatigue during prolonged flight. The visual entropy, eye fixation numbers, and eye fixation durations of ten novice pilots and ten expert pilots were analyzed for a four-hour simulated flight task consisting of four consecutive flight legs. Each flight leg lasted approximately one hour and contained five flight phases: takeoff, climb, cruise, descend, and landing. The pilots maneuvered the simulated B-52 aircraft following instrument flight rules (IFR) in a moderate-fidelity Microsoft Flight Simulator environment. Our results indicate that experts’ eye movement characteristics were significantly different from those of novices. In detail, novices' eye movements were more random, produced longer eye fixation durations, and had fewer eye fixation numbers on the areas of interest (AOIs) than the experts. In addition, the repetitive task (i.e., four consecutive flights) significantly impacted the eye movement characteristics for both experts and novices. Visual entropy and eye fixation duration increased, while eye fixation numbers decreased for both groups as the repetition index increased. Finally, the flight phases also affected eye movement characteristics. The results show that both experts' and novices' visual entropies were relatively higher during climb, cruise, and descend phases, whereas those were relatively lower during the takeoff and landing phases. The present results provide a foundation for us to better understand the similarities and dissimilarities of eye movement characteristics between the experts and novices for a prolonged flight. Lastly, potential scaffolding training methods and pilot anomaly alerting systems, derived from such eye movements, are introduced.

In plane sight: Inattentional blindness affects visual detection of external targets in simulated flight

Aviation places significant demands on pilots' perceptual and attentional capacities. The avoidance of other objects both on the ground and in the air is critical to safe flight. Research on automobile driving has revealed the occurrence of ‘inattentional blindness’ (IB) whereby objects clearly located within the visual field may not detected when drivers are concurrently engaged in another attention capturing task such as a cellphone conversation. Almost no comparable research has been conducted within the aviation domain despite the significance of both ground-based and mid-air collisions. The present study was designed to investigate the effects of diverting attentional resources away from the primary task of safely flying a simulated light aircraft from takeoff to cruising. Flight naïve students were trained to proficiency in a flight-simulator and flew two simulated flights with and without a competing attentional task. Detection of a variety of objects placed in the background was measured. The results showed that when distracted by an engaging cellphone conversation novice pilots failed to detect many of the objects located within the visual scene. Recognition accuracy was greater when pilots' attention was not diverted elsewhere. There was a reduction in time spent looking at some key flight instruments but not on others. Inattentional blindness poses significant flight safety risks and further research into both the stimulus and perceiver characteristics that promote or reduce inattentional blindness would be of significant benefit to aviation safety.

Effects of Curved and Flat Display Cockpit Systems on Visual Fatigue, Perceived Performance, and Gazing Behavior of Novice and Expert Pilots

Effects of Curved and Flat Display Cockpit Systems on Visual Fatigue, Perceived Performance, and Gazing Behavior of Novice and Expert Pilots

InDrone: a 2D-based drone flight behavior visualization platform for indoor building inspection

PurposeThe purpose of this research is to explore how to visually represent human decision-making processes during the performance of indoor building inspection flight operations using drones.Design/methodology/approachData from expert pilots were collected using a virtual reality drone flight simulator. The expert pilot data were studied to inform the development of an interactive 2D representation of drone flight spatial and temporal data – InDrone. Within the InDrone platform, expert pilot data were visually encoded to characterize key pilot behaviors in terms of pilots' approaches to view and difficulties encountered while detecting the inspection markers. The InDrone platform was evaluated using a user-center experimental methodology focusing on two metrics: (1) how novice pilots understood the flight approaches and difficulties contained within InDrone and (2) the perceived usability of the InDrone platform.FindingsThe results of the study indicated that novice pilots recognized inspection markers and difficult-to-inspect building areas in 63% (STD = 48%) and 75% (STD = 35%) of the time on average, respectively. Overall, the usability of InDrone presented high scores as demonstrated by the novice pilots during the flight pattern recognition tasks with a mean score of 77% (STD = 15%).Originality/valueThis research contributes to the definition of visual affordances that support the communication of human decision-making during drone indoor building inspection flight operations. The developed InDrone platform highlights the necessity of defining visual affordances to explore drone flight spatial and temporal data for indoor building inspections.

Learning to Assist Drone Landings

Unmanned aerial vehicles (UAVs) are often used for navigating dangerous terrains, however they are difficult to pilot. Due to complex input-output mapping schemes, limited perception, the complex system dynamics and the need to maintain a safe operation distance, novice pilots experience difficulties in performing safe landings in obstacle filled environments. In this work we propose a shared autonomy approach that assists novice pilots to perform safe landings on one of several elevated platforms at a proficiency equal to or greater than experienced pilots. Our approach consists of two modules, a perceptual module and a policy module. The perceptual module compresses high dimensionality RGB-D images into a latent vector trained with a cross-modal variational auto-encoder. The policy module provides assistive control inputs trained with the reinforcement algorithm TD3. We conduct a user study (n=33) where participants land a simulated drone with and without the use of the assistant. Despite the goal platform not being known to the assistant, participants of all skill levels were able to outperform experienced participants while assisted in the task.