High-fidelity Flight Simulator Research Articles

Enhanced robust adaptive flight control for a convertible VTOL UAV

In this paper, we propose less conservative formulations for candidate controllers of robust adaptive mixing control (RAMC) strategies. The RAMCs are employed to solve the trajectory tracking problem throughout the full flight envelope, with guaranteed stability, of a convertible plane (CP) vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV). Initially, the multi-body nonlinear dynamic model of the CP VTOL UAV is obtained using the Euler–Lagrange formalism, from which a linear parameter-varying (LPV) model is derived to be used in the design of the RAMC candidate controllers. The new formulations of the candidate controllers are proposed considering two approaches: a (i) Parallel Distributed Compensation (PDC); and (ii) a parameter-dependent Lyapunov function. Hardware-In-the-Loop (HIL) experiments are performed in a high-fidelity flight simulator to verify the fulfillment of real-time constraints, ensuring a computationally lightweight control strategy ready for implementation in a low-cost embedded computational system.

How flight experience impacts pilots’ decision-making and visual scanning pattern in low-visibility approaches: preliminary evidence from eye tracking

The visual approach is the most accident-prone phase of a flight, especially in low-visibility conditions. This preliminary study aimed to examine the effects of flight experience on pilots’ decision-making and visual scanning pattern in low-visibility approaches. Twenty pilots were separated into two groups based on their flight experience and completed the high- and low-visibility approaches in balanced order using a high-fidelity flight simulator. Pilots’ mental workload and visual scanning patterns were recorded via an eye tracker. The results showed that, compared to less flight-experienced pilots (20%, 3/15), experienced pilots (80%, 4/5) were more likely to make go-around decisions in the low-visibility approaches. Furthermore, they exhibited a more flexible and adaptable visual scanning pattern by quickly shifting their attention, as evidenced by decreased fixations and increased saccades. These findings suggest that the integration of visual scanning strategy and training solution with a marginally meteorological approach may enhance decision-making safety for novice pilots.

Towards ubiquitous and nonintrusive measurements of brain function in the real world: assessing blink-related oscillations during simulated flight using portable low-cost EEG.

Blink-related oscillations (BRO) are newly discovered neurophysiological phenomena associated with spontaneous blinking and represent cascading neural mechanisms including visual sensory, episodic memory, and information processing responses. These phenomena have been shown to be present at rest and during tasks and are modulated by cognitive load, creating the possibility for brain function assessments that can be integrated seamlessly into real-world settings. Prior works have largely examined the BRO phenomenon within controlled laboratory environments using magnetoencephalography and high-density electroencephalography (EEG) that are ill-suited for real-world deployment. Investigating BROs using low-density EEG within complex environments reflective of the real-world would further our understanding of how BRO responses can be utilized in real-world settings. We evaluated whether the BRO response could be captured in a high-fidelity flight simulation environment using a portable, low-density wireless EEG system. The effects of age and task demands on BRO responses were also examined. EEG data from 30 licensed pilots (age 43.37 +/- 17.86, 2 females) were collected during simulated flights at two cognitive workload levels. Comparisons of signal amplitudes were undertaken to confirm the presence of BRO responses and mixed model ANOVAs quantified the effects of workload and age group on BRO amplitudes. Significant increases in neural activity were observed post-blink compared to the baseline period (p < 0.05), confirming the presence of BRO responses. In line with prior studies, results showed BRO time-domain responses from the delta band (0.5-4 Hz) consisting of an early negative peak followed by a positive peak post-blink in temporal and parietal electrodes. Additionally, task workload and age-related effects were also found, with observations of the enhancement of BRO amplitudes with older age and attenuation of BRO responses in high workloads (p < 0.05). These findings demonstrate that it is possible to capture BRO responses within simulated flight environments using portable, low-cost, easy-to-use EEG systems. Furthermore, biological and task salience were reflected in these BRO responses. The successful detection and demonstration of both task-and age-related modulation of BRO responses in this study open the possibility of assessing human brain function across the lifespan with BRO responses in complex and realistic environments.

Towards a systems framework for the assurance of maritime autonomous systems

ABSTRACT The burgeoning adoption of Robotic and Autonomous Systems with Artificial Intelligence (RAS-AI) necessitates standardized approaches for Systems Engineering (SE) and assurance of RAS-AI systems. Unlike crewed systems, RAS-AI systems lack real-time human control, posing unique challenges yet to be addressed by existing regulations and SE frameworks. This paper proposes the ”Autonomous Delta” approach for the Assurance of RAS-AI systems (As4AI), focusing on Maritime Autonomous Systems (MAS). It comprises seven parts, contextualizing RAS-AI assurance in an Australian maritime context, providing key definitions, a literature review, and outlining the research methodology. The core of the paper (Part 5) presents an As4AI framework, integrating concepts from SE and high-fidelity flight simulation. A prototype instantiation at the Australian Institute of Marine Science confirmed the framework's utility and identified areas for refinement. The paper concludes with a summary and suggestions for future research enabling operational deployment of MAS specifically and autonomous systems in general.

High Fidelity Flight Simulation of Small Aircraft over Tall Buildings

High Fidelity Flight Simulation of Small Aircraft over Tall Buildings

The effect of working memory training on situation awareness in a flight simulator

The close relationship between working memory and situation awareness (SA) has been confirmed and further empirical investigations are lacking. The main aim of this study was to demonstrate the feasibility of working memory training for improving SA. Thirty-eight participants completed a challenging flight scenario in a high-fidelity flight simulator and were randomized into a training group (n = 20) or a control group (n = 18). The training group engaged in an adaptive dual N-back task for 2 weeks, while the control group was given a negative control task. Three-dimensional situation awareness rating technique (3D-SART) scores and situation awareness global assessment technique (SAGAT) scores were recorded to evaluate pretest and posttest SA. The results showed that both situational understanding dimension scores in the 3D-SART and SAGAT scores were significantly increased from the pretest to the posttest in the training group, while the control group showed no significant differences. It was concluded that working memory training can effectively improve individuals’ SA, which has important implication for future research.

Machine learning-based approach for identifying mental workload of pilots

In general, the fighter pilots are required to engage themselves entirely during flight operations involved in air-to-air combat while in the cockpit of a fighter aircraft. The performance has to be monitored continuously by classifying their cognitive workload levels during different phases of flying. Towards this direction, an experimental study was conducted in a realistic high-fidelity flight simulator environment to classify the Pilots’ Cognitive Workload (PCWL) level. A real-time implementation of algorithms to effectively organize the PCWL during takeoff, cruise and landing phases, physiological signals such as ECG and EEG of fighter pilots are used. The classification algorithms such as Linear Discriminant Analysis (LDA) classifier, Support Vector Machine (SVM) classifier, k-Nearest Neighbour (k-NN) classifier have been employed. It has resulted that takeoff (LDA – 75%, kNN – 60% and SVM – 75%) and landing phase (LDA – 75%, kNN – 60% and SVM – 75%) was better classified by HRV features while using PCA and cruise phase was classified better using EEG features (LDA – 72.44%, kNN – 62.92% and SVM – 59.02%) when PCA feature reduction technique was adopted. Using significant features by feature selection methods (PCA, statistically significant features) have shown improved classification accuracy compared to all the features classification method. The LDA and SVM are consistent classifiers compare to the kNN classifier. This study helps to classify the PCWL level at each flying phase due to increased task.

Validation of Aero-Load Estimator for Convair 880 Aircraft Flight Simulator Benchmark with Electro-Hydrostatic Actuators

Simulating an aircraft model using of high fidelity models of subsystems for its primary and secondary flight control actuators requires measuring or estimating aero-load data acting on flight control surfaces. One solution would be to incorporate the data recorded from flight tests, which is a time-consuming and costly process. This paper proposes another solution based on the validation of an aero-loads estimator or on the hinge moments predictor for fully electrical aircraft simulator benchmark. This estimator is based on an aerodynamic coefficient calculation methodology, inspired by Roskam’s method that uses the geometrical data of the wing and control surfaces airfoils. The hinge moment values are found from two-dimensional lookup tables where the deflections of the control surfaces, aircraft altitude, and aircraft angles of attack are the input vectors of the tables; and the resulting hinge moment coefficients are the output vectors. The resulting hinge moment coefficients of the Convair 880 primary flight control surfaces are compared to those of its recorded flight test data; the results from the new software solution were found to be very accurate. Hinge moment lookup tables are integrated in the Convair 880 high fidelity flight simulation benchmark using mathematical models of energy-efficient Electro-Hydrostatic Actuators (EHA). Autopilot controls are designed for the roll, pitch, attitude and yaw damper motions using Proportional Integral (PI) controller scheduled for different flight conditions. Several different aircraft simulation scenarios are evaluated to demonstrate the efficacy and accuracy of the predicted hinge moment results.

Integrated Pose Estimation Using 2D Lidar and INS Based on Hybrid Scan Matching.

Point cloud data is essential measurement information that has facilitated an extended functionality horizon for urban mobility. While 3D lidar and image-depth sensors are superior in implementing mapping and localization, sense and avoidance, and cognitive exploration in an unknown area, applying 2D lidar is inevitable for systems with limited resources of weight and computational power, for instance, in an aerial mobility system. In this paper, we propose a new pose estimation scheme that reflects the characteristics of extracted feature point information from 2D lidar on the NDT framework for exploiting an improved point cloud registration. In the case of the 2D lidar point cloud, vertices and corners can be viewed as representative feature points. Based on this feature point information, a point-to-point relationship is functionalized and reflected on a voxelized map matching process to deploy more efficient and promising matching performance. In order to present the navigation performance of the mobile object to which the proposed algorithm is applied, the matching result is combined with the inertial navigation through an integration filter. Then, the proposed algorithm was verified through a simulation study using a high-fidelity flight simulator and an indoor experiment. For performance validation, both results were compared and analyzed with the previous techniques. In conclusion, it was demonstrated that improved accuracy and computational efficiency could be achieved through the proposed algorithms.

Bayesian Nonparametric State-Space Model for System Identification with Distinguishable Multimodal Dynamics

The goal of system identification is to learn about underlying physics dynamics behind the time-series data. To model the probabilistic and nonparametric dynamics, Gaussian process (GP) has been widely used; GP can estimate the uncertainty of prediction. Traditional GP state-space models, however, are based on the Gaussian transition model, and thus they often have difficulty in describing more complex transition models, e.g., aircraft motions. To resolve the challenge, this paper proposes a framework using multiple GP transition models that is capable of describing multimodal dynamics. Furthermore, the model is extended to an information-theoretic framework, the so-called InfoSSM, by introducing a mutual information regularizer helping the model to learn interpretable and distinguishable multiple dynamics models. Two illustrative numerical experiments in a simple Dubins vehicle and high-fidelity flight simulator are presented to demonstrate the performance and interpretability of the proposed model. Finally, this paper further provides a use case of InfoSSM with Bayesian filtering for air traffic control tracking.

Cognitive Workload Analysis of Fighter Aircraft Pilots in Flight Simulator Environment

Maintaining and balancing an optimal level of workload is essential for completing the task productively. Fighter aircraft is one such example, where the pilot is loaded heavily both physically (due to G manoeuvering) and cognitively (handling multiple sensors, perceiving, processing and multi-tasking including communications and handling weapons) to fulfill the combat mission requirements. This cognitive demand needs to be analysed to understand the workload of fighter pilot. Objective of this study is to analyse dynamic workload of fighter pilots in a realistic high-fidelity flight simulator environment during different flying workload conditions. The various workload conditions are (a) normal visibility, (b) low visibility, (c) normal visibility with secondary task, and (d) low visibility with secondary task. Though, pilot’s flying performance score was good, the physiological measure like heart rate variability (HRV) features and subjective assessment (NASA-TLX) components are found to be statistically significant (p&lt;0.05) between tasks. HRV features such as SD2, SDNN, VLF and total power are found to be significant at all task load conditions. The features LFnu and HFnu are able to differentiate the effect of low visibility and secondary cognitive task, which was imposed as increased task in this study. This result benefits to understand the pilot’s task and performance at each flying phase and their cognitive demands during dynamic workload using HRV, which could assist pilot’s training schedule in optimal way on simulators as well as in actual flight conditions.

A Novel Method to Develop High Fidelity Laser Sensor Simulation Model for Evaluation of Air to Ground Weapon Algorithms of Combat Aircraft

Successful release of any air to ground weapon from a combat aircraft is determined based on the positional parameters received from the sensors and the mission cues. Laser designated pod is one of the most sought weapon sensor, which gives the accurate data for Air to Ground weapon aiming. Laser designated pod being hardware intensive system, works with real world environment, it increases the development and integration effort towards finalising the weapon aiming algorithms and also pilot vehicle interface requirements. A novel method using mathematical models and the atmospheric error models is proposed to develop a high fidelity laser designated pod simulation model for functional and performance evaluation of weapon algorithms. The factors affecting the weapon trajectory computations are also considered in the sensor model outputs. The sensor model is integrated in the high fidelity flight simulator, which consists of both aircraft and Real world systems either as actual or simulated for close loop pilot evaluation. The behaviour of the sensor model is cross validated and fine-tuned with the actual sensor output and confirmed that the developed laser designated pod sensor simulation model meets all the requirement to test the air to ground weapons in the flight simulator.

A hybrid approach for improving unsupervised fault detection for robotic systems

The use of robots in our daily lives is increasing. As we rely more on robots, thus it becomes more important for us that the robots will continue on with their mission successfully. Unfortunately, these sophisticated, and sometimes very expensive, machines are susceptible to different kinds of faults. It becomes important to apply a Fault Detection (FD) mechanism which is suitable for the domain of robots. Two important requirements of such a mechanism are: high accuracy and low computational-load during operation (online). Supervised learning can potentially produce very accurate FD models, and if the learning takes place offline then the online computational-load can be reduced. Yet, the domain of robots is characterized with the absence of labeled data (e.g., “faulty”, “normal”) required by supervised approaches, and consequently, unsupervised approaches are being used. In this paper we propose a hybrid approach - an unsupervised approach can label a data set, with a low degree of inaccuracy, and then the labeled data set is used offline by a supervised approach to produce an online FD model. Now, we are faced with a choice – should we use the unsupervised or the hybrid fault detector? Seemingly, there is no way to validate the choice due to the absence of (a priori) labeled data. In this paper we give an insight to why, and a tool to predict when, the hybrid approach is more accurate. In particular, the main impacts of our work are (1) we theoretically analyze the conditions under which the hybrid approach is expected to be more accurate. (2) Our theoretical findings are backed with empirical analysis. We use data sets of three different robotic domains: a high fidelity flight simulator, a laboratory robot, and a commercial Unmanned Arial Vehicle (UAV). (3) We analyze how different unsupervised FD approaches are improved by the hybrid technique and (4) how well this improvement fits our prediction tool. The significance of the hybrid approach and the prediction tool is the potential benefit to expert and intelligent systems in which labeled data is absent or expensive to create.

An Empirical Evaluation of Transfer-of-Training of Two Flight Simulation Games

Aim. The objective of this study was to collect evidence of transfer-of-training to professional performance provided by two stand-alone PC-based flight games. Background. These realistic games, Falcon 4.0 (F-16 specific) and Microsoft Flight Simulator (civil aircraft), are designed for entertainment purposes, lacking any purposeful or explicit instructional support. Method. This quasi-experimental study used three pre-existing groups of gamers (n = 37; Falcon 4.0 gamers, Microsoft Flight Simulator gamers and control group: gamers without flight game experience) that performed three typical F-16 flight tasks in a high-fidelity fixed-base flight simulator. Results. The Falcon 4.0 gamers performed substantially better on almost all tasks compared to the control group, and to a lesser degree to Microsoft Flight Simulator gamers. The Falcon 4.0 group showed near- and far-transfer on almost all flight performance measures: the game had prepared them for the generic and specific military aspects of the test flight tasks. Performance of the Microsoft Flight Simulator gamers indicated only far-transfer, i.e., transfer of more generic flight skills from the game to the test flight tasks. Conclusion. Both near- and far-transfer of job related competences may occur by playing realistic entertainment games.

Fundamentals of safety management: The Offshore Helicopter Transportation System Model

A precondition for a holistic, systemic and systematic management of offshore helicopter operations’ safety is an in-depth understanding of the corresponding system. This is essential because the components and the interactions between these components largely shape system and human performance, with the potential to create safety hazards and risks. However, current techniques of system component description in aviation are poorly specified, which contributes to a silo approach to hazard identification and risk analysis, poor safety management and missed safety targets. This paper addresses this shortcoming by developing a novel process to characterise the component-based system architecture of commercial air operations and implementing the process to create the Offshore Helicopter Transportation System Model. The paper discusses the theoretical justifications for generating such a system model and the shortcomings of the techniques currently recommended. In order to redress these, the process combines two task analysis techniques to derive an unbiased seminal system model, which is then compared to relevant models in the safety literature to ensure completeness. The process and resulting system model are validated by carefully selected subject matter experts during training sessions in high-fidelity flight simulators. The process can be applied to describe, by means of a system model, the activities of other industries in the aviation domain. Moreover, the model derived provides, for the first time, the necessary anchoring for future safety analyses in offshore helicopter operations.

Application of Quantitative Measures for Analysing Aircraft Handling Qualities

&lt;p&gt;The ease and precision with which pilot is able to handle the designated task determines the aircraft’s handling qualities. Accordingly, the most common methodology for determining aircraft’s handling qualities is through pilot opinions or through questionnaires. These subjective means of analysis is not reliable as the sole source of judgments. Quantitative metrics to analyse the task difficulty based on pilot’s performance, supplemented with subjective decision, can provide better insight into pilot workload levels and in turn the aircraft’s handling qualities. The application of few objective performance measurement techniques to flight data of a high performance fighter aircraft is discussed. Pilot/aircraft’s performance under different configurations is analysed. Analysis results show that pilots usually tend to give more priority to pitch axis in case of dual axis tracking task. And pilots are therefore more aggressive in accomplishing pitch axis tracking task than in roll. Workload assessments were also performed by comparing the results of single axis tracking experiments conducted using a high fidelity flight simulator with the flight data. It is seen that pilot’s aggressiveness levels in controlling the roll control inceptor is significantly less, with improved tracking accuracy when exercised as the primary task.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Defence Science Journal, Vol. 66, No. 1, January 2016, pp. 03-10, DOI: http://dx.doi.org/10.14429/dsj.66.9196&lt;/strong&gt;&lt;/p&gt;

Online data-driven anomaly detection in autonomous robots

The use of autonomous robots is appealing for tasks, which are dangerous to humans. Autonomous robots might fail to perform their tasks since they are susceptible to varied sorts of faults such as point and contextual faults. Not all faults can be known in advance, and hence, anomaly detection is required. In this paper, we present an online data-driven anomaly detection approach (ODDAD) for autonomous robots. ODDAD is suitable for the dynamic nature of autonomous robots since it declares a fault based only on data collected online. In addition, it is unsupervised, model free and domain independent. ODDAD proceeds in three steps: data filtering, attributes grouping based on dependency between attributes and outliers detection for each group. Above a calculated threshold, an anomaly is declared. We empirically evaluate ODDAD in different domains: commercial unmanned aerial vehicles (UAVs), a vacuum-cleaning robot, a high-fidelity flight simulator and an electrical power system of a spacecraft. We show the significance and impact of each component of ODDAD . By comparing ODDAD to other state-of-the-art competing anomaly detection algorithms, we show its advantages.

Efficient complete coverage of a known arbitrary environment with applications to aerial operations

The problem of coverage of known space arises in a multitude of domains, including search and rescue, mapping, and surveillance. In many of these applications, it is desirable or even necessary for the solution to guarantee both the complete coverage of the free space, as well as the efficiency of the generated trajectory in terms of distance traveled. A novel algorithm is introduced, based on the boustrophedon cellular decomposition technique, for computing an efficient complete coverage path for a known environment populated with arbitrary obstacles. This hierarchical approach first partitions the space to be covered into non-overlapping cells, then solves the Chinese postman problem to compute an Eulerian circuit traversing through these cells, and finally concatenates per-cell seed spreader motion patterns into a complete coverage path. Practical considerations of the coverage system are also explored for operations with a non-holonomic aerial vehicle. The effects of various system parameters are evaluated in controlled environments using a high-fidelity flight simulator, in addition to over 200 km of in-field flight sessions with a fixed-wing unmanned aerial vehicle.

An Integrative Approach to Understanding Flight Crew Activity

In this paper, we describe an integrative approach to understanding flight crew activity. Our approach combines contemporary innovations in cognitive science theory with a new suite of methods for measuring, analyzing, and visualizing the activities of commercial airline flight crews in interaction with the complex automated systems found on the modern flight deck. Our unit of analysis is the multiparty, multimodal activity system. We installed a variety of recording devices in high-fidelity flight simulators to produce rich, multistream time-series data sets. The complexity of such data sets and the need for manual coding of high-level events make large-scale analysis prohibitively expensive. We break through this analysis bottleneck by using our newly developed integrated software system called ChronoViz, which supports visualization and analysis of multiple sources of time-coded data, including multiple sources of high-definition video, simulation data, transcript data, paper notes, and eye gaze data. Four examples of flight crew activity serve to illustrate the methods, the theory, and the kinds of findings that are now possible in the study of flight crew interaction with flight deck automation.

Complementary Information Distribution to Improve Team Performance in Military Helicopter Operations: An Experimental Study

This study compared a complementary information distribution, that is, purposeful distribution to individual crewmembers of the information needed for their individual tasks, to the current distribution where all crewmembers can access all information in the flight deck. The complementary information distribution is intended to foster a complementary mental model that promotes each team member's access to information according to his or her immediate tasks. The hypothesis—that the complementary information distribution would improve performance and team interaction—was tested in a high-fidelity military helicopter flight simulator with 20 Army aviators. The complementary information distribution was found to have no effect or a positive effect (and no negative effects) on measures of individual's task performance and error rates, with greater positive effects on most measures in off-nominal compared to nominal flight conditions. In addition, measures of team communication and awareness of fellow crewmembers' information needs and workload also found only no effect or positive effects with a complementary information distribution.