Cockpit Simulator Research Articles

Experimental study on visual ergonomics of an aircraft cockpit considering an extremely wide range of illuminance conditions

Vision is the main way for pilots to obtain information, and good visual ergonomics are an important support for ensuring aircraft flight safety. The range of illumination changes in the light environment of the aircraft cockpit is very wide, and research on the visual ergonomics of the cockpit needs to consider various extreme lighting conditions. This study conducted visual ergonomics experiments on 15 participants in a full-scale simulated cockpit, examining the accuracy, reaction time, and subjective evaluation of visual tasks under 8 typical environmental lighting intensity levels. The experimental results show that, except for head-up display, the accuracy of visual target interpretation tasks performed by other display devices under different brightness conditions remains at a high level. And as the brightness of the display device increases, the accuracy of interpretation gradually increases, and the reaction time gradually decreases. In terms of subjective evaluation, there is a significant correlation between fuzziness, fatigue, clarity of image symbols, resolution between symbols, comfort of the image, and overall satisfaction with the image, but the correlation with environmental illumination level is relatively low. The experimental results can provide a certain theoretical basis for the design of cockpit lighting environment.

Mathematical Model of HoloLens 2 Goggles Kinematics for AR Support of Flight Simulator

The paper describes how the kinematics model of HoloLens 2 goggles mounted on the user’s head is used for converting the reference frame fixed to the flight simulator with the reference frame fixed to the goggles. The constant correspondence between the two has to be kept to properly map the internal 3D space relative to the position of the device with the real 3D space of the flight simulator cockpit in which HoloLens 2 produces holographic objects. Such conversions are prerequisites of SLAM (Simultaneous Localization and Mapping) algorithms which are used by AR devices to build a map of an environment, and, at the same time, this map is used to compute the goggle’s own position and orientation. In the paper, we also present the application of HoloLens 2 goggles in the WrightBroS project to support flight simulator users (pilots and/or technicians) in their training and maintenance procedures. Keywords: augmented reality (AR), slam, kinematics, hololens 2, flight simulators

Taking into account a location of aircraft’s center of mass during motion cuing

Compared to other information sources, motion cues provide a pilot with anticipatory information about spatial position and movement of aircraft. For motion cueing a flight simulator cockpit is installed on a motion system, movement of which motion cueing. Therefore, motion system is one of the most important components of full flight simulators. The problem of effective use of constructive resources of six-degrees of freedom synergistic motion system has been solved. But the problem of improving the motion cueing remained unsolved, due to the fact that location of motion system center of rotation is significantly different from location of aircraf’s center of gravity, and motion cues differ from real, flight one. The study subject is motion cueing on flight simulators. The problem was solved on the basis of simplified operator for transformation of motion system movements along individual degrees of freedom into jack movements, cubic spline functions to describe the dependence of the centers of rotation along pitch and yaw, and optimization theory using the deformable polyhedron method. The formulated and solved problem of taking into account of location of aircraft’s center of gravity during motion cueing along pitch and yaw increases an efficiency of using of constructive resource of a six-degrees of freedom synergistic motion system, a motion cueing fidelity and training realism on flight simulator.

Design and Analysis of the Wing Functioning as a Ramp for a Spitfire Flight Simulator

This paper designed and analysed the wing for a Spitfire Mk9 Flight Simulator using three-dimensional modeling, finite element analysis and fatigue analysis methodologies with 3D EXPERIENCE, ABAQUS/FE-SAFE software. The results demonstrated that by using the wing structure as the access ramp to the cockpit simulator, the accessibility, safety and structural integrity could be improved vastly in comparison with the existing Spitfire flight simulators.

Optimum display luminance under a wide range of ambient light for cockpit displays.

The self-luminous cockpit displays need to be adaptive to a wide range of ambient light levels, which changes from very low illuminance to very high levels. Yet, current studies on evaluation and luminance setting of displays in bright surroundings are still limited. In this study, a three-dimensional visual ergonomic experiment was carried out to investigate how bright a cockpit display should be to meet aircrew operational requirements under different illuminance. A lab study with a within-subjects (N = 12) design was conducted in a simulated cockpit. According to the Weber-Fechner's Law, human observers evaluated five display luminance conditions (101, 101.5, 102, 102.5, 103 cd/m2) under five ambient illuminance conditions (10°, 101, 102, 103, 104 lx). Visual performance, visual fatigue and visual comfort were used as evaluation bases, which were measured by d2 task, subjective fatigue questionnaire and visual perception semantic scales. Nonlinear function fitting was used to calculate the optimal luminance under a certain illuminance. Finally, curvilinear regression was used to analyze the illuminance and its corresponding optimal luminance. Based on Silverstein luminance power function, a luminance adjustment model with the form of power function was obtained. The proposed three-dimensional model fits the experimental data well and is consistent with the existing studies. It can be regarded as a supplement and optimization of the previous model under high ambient illuminance. This study can contribute not only to the pleasing luminance setting of panel displays in aircraft cockpits but also to other self-luminous devices, such as tablet devices, outdoor monitoring equipment and advertising screens.

Toward passive BCI: asynchronous decoding of neural responses to direction- and angle-specific perturbations during a simulated cockpit scenario

Neuroimaging studies have provided proof that loss of balance evokes specific neural transient wave complexes in electroencephalography (EEG), called perturbation evoked potentials (PEPs). Online decoding of balance perturbations from ongoing EEG signals can establish the possibility of implementing passive brain-computer interfaces (pBCIs) as a part of aviation/driving assistant systems. In this study, we investigated the feasibility of identifying the existence and expression of perturbations in four different conditions by using EEG signals. Fifteen healthy participants experienced four various postural changes while they sat in a glider cockpit. Sudden perturbations were exposed by a robot connected to a glider and moved to the right and left directions with tilting angles of 5 and 10 degrees. Perturbations occurred in an oddball paradigm in which participants were not aware of the time and expression of the perturbations. We employed a hierarchical approach to separate the perturbation and rest, and then discriminate the expression of perturbations. The performance of the BCI system was evaluated by using classification accuracy and F1 score. Asynchronously, we achieved average accuracies of 89.83 and 73.64% and average F1 scores of 0.93 and 0.60 for binary and multiclass classification, respectively. These results manifest the practicality of pBCI for the detection of balance disturbances in a realistic situation.

Influence of Estimated Mission Data on Air Mission Commander Situation Awareness During Datalink Loss

AbstractThis article presents the impact of displaying accurate or inaccurate position estimates on air mission commanders’ (AMC) situation awareness (SA) during datalink interruptions between AMC and unmanned aerial vehicles (UAVs). The lack of current information about UAV state and position during datalink interruptions is suspected to negatively influence the controlling AMC's SA. We assume that the AMC's SA can be supported by displaying a state estimate, based on last transmitted data, instead of displaying last known position. An experimental evaluation in a full‐mission research cockpit simulator showed improvements in subjective SA with estimate shown. Objective SA only improved, if the estimate was accurate. Although we expected a negative impact of inaccurate estimates on objective SA (since the user might falsely trust it), none could be observed. Improvements in SA with accurate estimates, but no observable degradations with inaccurate estimates suggest benefits of showing an estimate in any case.

A Socio-Technical Simulation Model for the Design of the Future Single Pilot Cockpit: An Opportunity to Improve Pilot Performance

The future deployment of single pilot operations must be supported by new cockpit computer services. Such services require an adaptive context-aware integration of technical functionalities with the concurrent tasks that a pilot must deal with. Advanced artificial intelligence supporting services and improved communication capabilities are the key enabling technologies that will render future cockpits more integrated with the present digitalized air traffic management system. However, an issue in the integration of such technologies is the lack of socio-technical analysis in the design of these teaming mechanisms. A key factor in determining how and when a service support should be provided is the dynamic evolution of pilot workload. This paper investigates how the socio-technical model-based systems engineering approach paves the way for the design of a digital assistant framework by formalizing this workload. The model was validated in an Airbus A-320 cockpit simulator, and the results confirmed the degraded pilot behavioral model and the performance impact according to different contextual flight deck information. This study contributes to practical knowledge for designing human-machine task-sharing systems.

Simulation Technology in Anesthesiology.

Simulation Technology in Anesthesiology.

Remote driving technology based on 5G

Relying on the 5G development wave, this project realizes human-vehicle interaction in the simulated cockpit in an innovative way, and remotely controls vehicle-like robots (unmanned vehicles). It aims to build a vehicle remote control system with a lower cost and a more stable way, and develop a remote driving robot based on 5G communication for distribution, disinfection or more precise tasks.

Operator State Estimation to Enable Adaptive Assistance in Manned-Unmanned-Teaming

With the continued development of unmanned aerial vehicle (UAV) technologies, the UAV on-board automation is increasingly more capable of performing tasks formerly done by human operators. Thereby, the role of UAVs is changing from being mere tools to become members of integrated manned-unmanned systems. However, the high automation necessary to achieve this cooperation, introduces a new set of negative effects for the human operator such as complacency or automation bias. Adaptive assistance is one approach to counteract these negative effects seen in human-automation-interaction. To enable adaptive assistance, we present a cognitive state estimation framework for a MUM-T aircraft application. The goal of this approach is to assess attention allocation and SA of a pilot in real-time and identify possible breakdowns in the situational picture that could cause performance decrements and errors. The design of a MUM-T cockpit simulator is presented to describe how this cognitive state estimation framework is integrated into a human-autonomy-teaming environment. The results of initial simulator experiments are presented and areas of further research are identified.

Flying with a Compass

I met VC in an inpatient progress note. I felt I was standing by the window, watching the attending on rounds come and sit by the bed of this dying retired family doctor. I could almost hear the pleasant, unhurried conversation about VC’s life and what he wanted from what remained of it. The note immortalized in the electronic health record was written with poetic brevity and it was the most beautiful, succinct, and thorough description of goals of care I’d ever seen.“Long discussion with VC about his life, passions, and goals of care,” the note started. Autonomy and choice were important to him, as were his family and relationships. He wanted to avoid death in an institutional setting, having spent plenty of time in hospitals during his medical career. His goal was to go home soon, safely, and as comfortably as possible. The note expressed his joy in being a husband, father, physician, and grandfather and in watching the astounding innovations in medicine that had happened during his career.VC looked tired when I arrived to introduce myself as the anesthesiologist caring for him for his PleurX catheter placement. He paused every word or two to pull oxygen from his nasal cannula. His chart specified he was do not resuscitate/do not intubate. My main goal was to confirm my understanding of his wishes and define what airway and hemodynamic support interventions he would accept if I deemed them appropriate and what he would refuse under any circumstances. I wanted to avoid exhausting him with a lengthy conversation. Pausing to speak meant he would need to work to catch up on oxygenation afterward. I described what I had read in the progress note and he was able to nod and interject a few words, communicating that his perioperative care preferences were very consistent with the overarching goals he had conveyed before.Anesthesiologists frequently care for patients near the end of life. One of the great privileges of my job is assisting in palliative procedures that, as in VC’s case, allow the patient to leave the hospital and spend the rest of their time at home or, if unable to leave, then spend it in greater comfort and peace. When the procedure’s goals, the patient’s wishes, and the acceptable perioperative procedures align, I feel only joy.My time with most of my patients is brief. I rely on documentation from established therapeutic relationships to help me understand the individuals who are about to entrust me with their lives. The more I know, the better I’m able to care for them and the more trust I’m able to gain in the brief conversation we have in our busy preoperative units.The primary care provider who had this conversation and wrote this note has my deepest gratitude. The earlier these conversations start, and the more often they start in the setting of an established therapeutic relationship, the more they enrich later acute or subspecialty care. Not only did the note’s author engage in this vital conversation and clearly document it, but also placed it in a location which underlined its importance. The electronic health record is very successful in capturing billable care, improving compliance, and producing notes Homerian in length. It is also a powerful and underutilized communication tool. The goals of care were at the top of his progress note, the first thing I saw when I was reviewing VC’s chart.Anesthesiology is often compared with the airline industry—we have preprocedure checklists and do careful equipment checks. We sit in a simulated cockpit, getting constant feedback from our instruments and monitoring equipment. Induction, maintenance, and emergence mirror takeoff, cruising altitude, and landing. Yet all of this is secondary to knowing where you’re going. Eliciting and documenting priorities, wishes, and goals of care in the setting of a high-quality primary care relationship gives us a magnetic north to navigate by in these challenging cases.

Evaluation of a Tunnel-in-the-Sky Head-Up Display Design for Curved Approaches Using Eye-Tracking

Abstract. Head-up displays (HUD) assist pilots, especially in the approach and landing phase. In this paper we compared pilots’ eye-tracking behavior between a pathway-in-the-sky layout versus the more conventional two-dimensional symbology in an HUD in a fixed-based cockpit simulator. In particular, we wanted to assess how visual attention was distributed within each layout (tunnel and standard). Performance and eye-tracking data were recorded, as well as workload and situation awareness measures. Results showed that the tunnel-in-the-sky symbology provided a very high tracking performance with low perceived workload and high perceived situational awareness. The analysis of the eye-tracking data revealed possible attentional tunneling with the tunnel-in-the-sky symbology and large differences between the displays in the distribution of visual attention.

Speech recognition in flight simulator

The article is devoted to the investigation of speech recognition in Flight Simulator cockpit. We have done research and developed software for speech recognition in Flight Simulator with limited vocabulary in C# from scratch. The speech recognition system works in real time and offline mode in Windows platform. We used Mel Frequency Cepstral Coefficients and Linear Predictive Coefficients feature extraction algorithms and trained the system by multilayer artificial neural networks. The User Interface of platform is highly functional and allows users to update system parameters through the interface. Our Speech Recognition system results are compared to the results of Microsoft Speech SDK and received satisfactory achievement.

Effects of Automation Reliability and Trust on System Monitoring Performance in Simulated Flight Tasks

The current study was designed to empirically examine the effects of trust and automation reliability on multi-tasking performance in a simulated cockpit setting using the Multi-Attribute Task Battery II (MATB-II). The MATB-II simulates tasks often performed by pilots in-flight, tasking the operator with attending to automated systems and correcting errors when they inevitably occur. Over the course of three 30-minute trials, two levels of automation reliability were presented in the current study (R50% and R90%). It was hypothesized that automation reliability and trust would affect both workload and performance in this multi-tasking environment. Results indicated that reliability significantly affected monitoring performance on the MATB-II. More reliable automation resulted in poorer monitoring performance, while trust appeared to have little impact. These results provide further evidence for how operators trust and interact with automation, a topic that is relevant to the implementation of automated systems in a variety of human-machine systems such as aviation.

Three-Dimensional Measurement Applied in Design Eye Point of Aircraft Cockpits.

Inappropriate design eye point (DEP) will lead to nonstandard sitting postures, including nonneutral head positions and other uncomfortable sitting postures, which are high risk factors for neck pain in fighter pilots exposed to high G forces. Therefore, application of a 3D measurement method to collect data regarding eye position while in the cruising sitting posture in the aircraft cockpit to guide the design eye point has been proposed. A total of 304 male fixed wing aircraft pilots were divided into two groups. Subgroup A (N = 48) were studied to define the cruising posture during flight. Subgroup B (N = 256) were studied with Romer 3D measurement equipment to locate the cruising eye position of the pilots in a simulated cockpit. The 3D data were compared to DEP data in the current standard cockpit. According to 3D measurement, the vertical distance from the cruising eye point to the neutral seat reference point was 759 mm, which is 36 mm lower than that of the Chinese standard DEP and also lower than the U.S. military standard. The horizontal distance was 131 mm, which is 24 mm shorter than that of the Chinese standard. The current DEP data cannot fulfill the needs of fighter pilots and should be amended according to the results of the 3D measurement so that pilots can acquire the optimal cruising posture in flight. This new method has the value of practical application to investigate cockpit ergonomics and the measurement data can guide DEP design.Wang Y, Guo X, Liu Q, Xiao H, Bai Y. Three-dimensional measurement applied in design eye point of aircraft cockpits. Aerosp Med Hum Perform. 2018; 89(4):371-376.

Learning flight procedures by enacting and receiving feedback

Learning flight procedures is part of any pilot training. The conventional learning method consists in learning and practicing the procedure written on a sheet of paper along with printed images of the cockpit. The purpose of the present paper was to test the efficiency of a tactile interactive multimedia training tool designed to foster the self-regulated learning of flight procedures, especially through enacting relevant gestures and providing feedback. Results showed that learning with this tool did not lead to significant shorter learning times than with the conventional learning. However, on a delayed retention test in a real A320 cockpit simulator, learners of the experimental group performed the procedure more rapidly than those of the control group. Results suggested that a training tool that incites learners to perform similar gestures than those in the real environment and that provides feedback, helped learners to transform declarative into procedural knowledge.

An unmanned aircraft "see and avoid" algorithm development platform using opengl and opencv

This work presents a see-and-avoid (SAA) system for autonomous Unmanned Aircraft Systems (UAS) and a testbed for evaluating SAA capabilities using simulated cockpit video generated using the comput...

Tactile cueing with active cyclic stick for helicopter obstacle avoidance: development and pilot acceptance

As helicopters land and takeoff in obstructed areas and on unprepared landing sites the risk of a collision with an obstacle is high. DLR have implemented a function to provide haptic feedback on the controls of a helicopter as a haptic pilot assistance for obstacle avoidance in order to evaluate its’ benefits for flying in obstructed areas. The cueing forces are provided in the direction away from the obstacle, and the intensity is a function of the distance to the obstacle. Two different force strategies were prepared for evaluation. The first was a steady, continuously increasing counter force in combination with an increasing spring gradient. The second was a repetitive, rectangular shaped pulsing force with increasing frequency and amplitude. As a proof of concept three EC135 pilots experienced in helicopter emergency medical service (HEMS) assessed the acceptability of such a system for operational use in the EC135 ACT/FHS simulator cockpit in DLR’s Air Vehicle Simulation Center (AVES). It achieved a high level of acceptance. This paper presents the development and evaluation results and gives recommendations for the further development.

Weather display symbology affects pilot behavior and decision-making

ObjectiveTo examine the basic discriminability of weather display symbols and to assess the effects of weather display symbology on pilot behavior and decision-making. MethodDuring a cockpit simulation, 24 instrument-rated general aviation (GA) pilots were randomly allocated to one of three simulation groups. Pilots flew a Cessna 172 single-engine GA aircraft under Visual Meteorological Conditions (VMC) and Instrument Meteorological Conditions (IMC) while avoiding hazardous weather. We manipulated the weather display so that each pilot group used a different weather symbology (i.e., symbols and colors). We measured dependent variables for weather avoidance, communication, weather display usage, and cognitive engagement (i.e., oxygenation from Functional Near-Infrared [fNIR] measures). During a change-detection experiment, 20 naïve participants performed a detection task of changes in Aviation Routine Weather Reports (METARs), precipitation, Significant Meteorological Information (SIGMET), lightning, and time-stamp images. ResultsThe simulation outcome showed credible pilot group differences in weather deviations, cognitive engagement, and weather display usage. The change-detection experiment revealed credible differences in discriminability of METAR, SIGMET, and lightning symbols. ConclusionSymbol and color variations in weather displays contribute to perceptual asymmetries which affect pilot behavior and decision-making. We recommend the development of cockpit applications that use weather data to automatically track hazardous conditions and alert the pilot of potential weather conflicts or weather changes.