Synthetic Vision Research Articles

Terrain Elevation Data Integrity Monitor and Referenced Navigation

Synthetic vision systems provide a synthesized view of the outside world to pilots on displaysintheflightdeck.Terraindatabasesonboardaircraftareusedasthesourceofterrain elevation information on synthetic vision system displays. Because the primary function of thesedisplaysistoimproveflightsafety,itisimperativethattheterraindatausedtogenerate the imagery conform to a high level of integrity. Otherwise, instead of preventing accidents, the terrain database would be cause of more. Hence, it is necessary to include an integrity monitor function that ensures the terrain data are consistent with the real world. In this paper, we revisit previously proposed concepts and present the development of a 3D spatial data integrity monitor. The framework is also extended to a terrain referenced navigation scheme such that both operations can be performed simultaneously. Presented furthermore is a Kalman filter design and its associated tradeoffs for improving the performance of the integrity monitor and navigation functions. The performance of the integrity monitor and position estimator for navigation is evaluated using flight test data collected in the vicinity of Braxton County airport in West Virginia.

Evaluating the Effects of Dimensionality in Advanced Avionic Display Concepts for Synthetic Vision Systems

Synthetic vision systems provide an in-cockpit view of terrain and other hazards via a computer-generated display representation. Two simulator-based experiments examined several display concepts for synthetic vision and evaluated how such displays modulate pilot performance. Experiment 1 (24 general aviation pilots) compared 3 navigational display (ND) concepts: 2D coplanar, 3D, and dual-perspective. Experiment 2 (12 commercial airline pilots) evaluated baseline “blue sky–brown ground” or synthetic-vision-enabled primary flight displays (PFDs) and 3 ND concepts: 2D coplanar with and without synthetic vision and a dynamic multimode rotatable exocentric format. In general, the results pointed to an overall advantage for a dual-perspective format, whether it be stand-alone (Experiment 1) or available via rotatable viewpoints (Experiment 2). Furthermore, Experiment 2 revealed benefits associated with utilizing synthetic vision in both the PFD and ND representations and the value of combined ego- and exocentric presentations.

Synthetic Vision Enhanced Surface Operations With Head-Worn Display for Commercial Aircraft

Efficiency and safety of airport surface operations can be enhanced by using synthetic vision and associated technologies, employed on a head-up display (HUD) combined with a head-down display electronic moving map. Past research has noted that 2 major limitations of HUDs during ground operations were its monochrome form and its limited, fixed field of regard. These limitations can be overcome with the use of a head-worn display (HWD). This article describes the results of a simulation experiment that showed that a fully integrated HWD concept provided significantly improved path performance compared to using paper charts alone. When comparing the HWD and HUD concepts, there were no differences found for path performance or subjective ratings of workload and marginally significant differences found for reported situation awareness and pilot comments of perceived system safety. Implications and directions for future research are described.

Commercial Flight Crew Decision Making During Low-Visibility Approach Operations Using Fused Synthetic and Enhanced Vision Systems

The use of an enhanced vision (EV) system in civil aircraft is projected to increase rapidly as the Federal Aviation Administration recently changed the aircraft operating rules under Part 91, revising the flight visibility requirements for conducting operations to civil airports. Operators conducting straight-in instrument approach procedures may now operate below the published approach minimums when using an approval EV system that shows the required visual references on the pilot's Head-Up Display (HUD). An experiment was conducted to evaluate the complementary use of synthetic vision (SV) and EV technologies, focusing on new techniques for integration and/or fusion of synthetic and enhanced vision technologies under the newly adopted FAA rules. The current paper describes a fixed-based simulation experiment that evaluated several display concepts located on the pilot-flying HUD and pilot-not-flying auxiliary display. Results specific to flight crew response, during non-normal events and below minimum visibility, using the fused synthetic/enhanced vision system are discussed here. The results demonstrated that the integrated SV/EV display did not adversely impact detection of lateral navigational solution errors on runway incursions. Additionally, flight crews were able to effectively use existing EV system procedures in their decision-making but found the visual transition from an excellent EV image to natural vision by 100 ft height above touchdown to be awkward. Finally, the addition of scene-linked symbology significantly enhanced flight crew go-around decision-making and reduced the number of illegal landings during approaches in which the required visual references to continue the landing were unavailable.

Attentional Tunneling and Task Management in Synthetic Vision Displays

This article discusses attentional tunneling as one cause of breakdowns in task management. The phenomenon is defined, and it is shown how it can be induced by head-up display location, and compelling 3-dimensional perspective displays. The results of 6 experiments using a synthetic vision display, with or without a highway in the sky (HITS) display are then summarized, as these reveal the attention tunneling phenomenon manifest in the failure to detect unexpected air hazards. Two factors that enhance this phenomenon are identified: location away from the center of visual scan and the presence of the HITS.

Three‐dimensional statistical modeling of neuronal populations: Illustration with spatial localization of supernumerary neurons in the locus coeruleus of quaking mutant mice

An algorithm for the three-dimensional statistical representation of neuronal populations was designed and implemented. Using this algorithm a series of 3D models, calculated from repeated histological experiments, can be combined to provide a synthetic vision of a population of neurons taking into account biological and experimental variability. Based on the point process theory, our algorithm allows computation of neuronal density maps from which isodensity surfaces can be readily extracted and visualized as surface models revealing the statistical organization of the neuronal population under study. This algorithm was applied to the spatial distribution of locus coeruleus (LC) neurons of 30- and 90-day-old control and quaking mice. By combining 12 3D models of the LC, a region of the nucleus in which a subpopulation of neurons loses its noradrenergic phenotype between 30 and 90 days postnatally was demonstrated in control mice but not in quaking mice, leading to the hyperplasia previously reported in adult mutants. Altogether, this algorithm allows computation of 3D statistical and graphical models of neuronal populations, providing a contribution to quantitative 3D neuroanatomical modeling.

Synthetic Vision: Application Areas, Rationale, Implementation, and Results

Synthetic vision is regarded as a means to increase both safety and operational capabilities. The design of a synthetic vision system presents the designer with questions regarding which data need to be presented, how the data should be represented, and how the representation should be mapped onto the display. A prototype synthetic vision system was designed and iteratively refined. This article discusses how the results from research into spatial displays have been used to understand the design options and resulted in the choices made. Also, an overview of the flight tests, their specific goals, and the main results is presented.

Combining Enhanced and Synthetic Vision for Autonomous All-Weather Approach and Landing

This article is about the design and evaluation of the human–machine interface of an enhanced and synthetic vision system. Several tunnel-in-the-sky and pathway-in-the-sky concepts including scene-linking elements have been evaluated. Pathway-in-the-sky displays enable pilots to accurately fly difficult trajectories. However, these displays may drive pilots' attention to the aircraft guidance task at the expense of other tasks, particularly when the pathway display is located head-down. A pathway head-up display (HUD) might be a viable solution to overcome this disadvantage. Moreover, the pathway might mitigate the perceptual segregation between the static near domain and the dynamic far domain and hence might improve attention switching between both sources. To more comprehensively overcome the perceptual near-to-far domain disconnect, alphanumeric symbols could be attached to the pathway leading to an HUD design concept called scene linking. A scene-linked pathway-predictor concept was implemented on a monocular retinal scanning head-mounted device (HMD) in combination with an optical head tracker. The evaluation comprises low-fidelity part-task simulations, high-fidelity simulator runs, and flight trials. Where laboratory experiments found evidence in favor of scene-linked pathway HUDs or HMDs, the real flight tests could not fully support this display concept. Even so, in all studies evidence has been found that the head-up pathway concept could be superior to current head-up solutions.

Trade-Offs in Synthetic Vision System Display Resolution, Field of Regard, Terrain Data Density, Texture, and Shading During Off-Path Operations

We conducted 2 independent fixed-base flight simulator studies to determine trade-offs in synthetic vision system (SVS) display resolution, field of regard, terrain data density, texture, and shading during off-path operations. The data we present herein are a synthesis of prior work and may be useful for consideration by SVS designers and certification entities. Pilots flew off-path tasks, following the middle of a curved valley depicted on the SVS. The SVS parameters were manipulated between runs and cross-track deviations from the desired path following the middle of the valley were recorded. Off-path terrain navigation tasks were used to isolate the effect of the different SVS terrain depictions on cross-track error without the overriding effects of other flight guidance symbology. For off-path SVS operations and based on the results presented in this article, we recommend an SVS head-down display resolution of about 105 pixels per inch (ppi; 32.7 pixels per degree of visual angle) or better at a viewing distance of 21 in., a field of regard of 60 degrees, a terrain data density of 6 arc sec or better, a two-color checkerboard or elevation-based coloring scheme (or a combination thereof), and Gouraud shading of the texture.

Task-Specific Codes for Face Recognition: How they Shape the Neural Representation of Features for Detection and Individuation

BackgroundThe variety of ways in which faces are categorized makes face recognition challenging for both synthetic and biological vision systems. Here we focus on two face processing tasks, detection and individuation, and explore whether differences in task demands lead to differences both in the features most effective for automatic recognition and in the featural codes recruited by neural processing.Methodology/Principal FindingsOur study appeals to a computational framework characterizing the features representing object categories as sets of overlapping image fragments. Within this framework, we assess the extent to which task-relevant information differs across image fragments. Based on objective differences we find among task-specific representations, we test the sensitivity of the human visual system to these different face descriptions independently of one another. Both behavior and functional magnetic resonance imaging reveal effects elicited by objective task-specific levels of information. Behaviorally, recognition performance with image fragments improves with increasing task-specific information carried by different face fragments. Neurally, this sensitivity to the two tasks manifests as differential localization of neural responses across the ventral visual pathway. Fragments diagnostic for detection evoke larger neural responses than non-diagnostic ones in the right posterior fusiform gyrus and bilaterally in the inferior occipital gyrus. In contrast, fragments diagnostic for individuation evoke larger responses than non-diagnostic ones in the anterior inferior temporal gyrus. Finally, for individuation only, pattern analysis reveals sensitivity to task-specific information within the right “fusiform face area”.Conclusions/SignificanceOur results demonstrate: 1) information diagnostic for face detection and individuation is roughly separable; 2) the human visual system is independently sensitive to both types of information; 3) neural responses differ according to the type of task-relevant information considered. More generally, these findings provide evidence for the computational utility and the neural validity of fragment-based visual representation and recognition.

Synthetic Vision Displays for Planetary and Lunar Lander Vehicles

Aviation research has demonstrated that Synthetic Vision (SV) technology can substantially enhance situation awareness, reduce pilot workload, improve aviation safety, and promote flight path control precision. SV, and related flight deck technologies are currently being extended for application in planetary exploration vehicles. SV, in particular, holds significant potential for many planetary missions since the SV presentation provides a computer-generated view for the flight crew of the terrain and other significant environmental characteristics independent of the outside visibility conditions, window locations, or vehicle attributes. SV allows unconstrained control of the computer-generated scene lighting, terrain coloring, and virtual camera angles which may provide invaluable visual cues to pilots/astronauts, not available from other vision technologies. In addition, important vehicle state information may be conformally displayed on the view such as forward and down velocities, altitude, and fuel remaining to enhance trajectory control and vehicle system status. The paper accompanies a conference demonstration that introduced a prototype NASA Synthetic Vision system for lunar lander spacecraft. The paper will describe technical challenges and potential solutions to SV applications for the lunar landing mission, including the requirements for high-resolution lunar terrain maps, accurate positioning and orientation, and lunar cockpit display concepts to support projected mission challenges.

Testing the Efficacy of Synthetic Vision during Non-Normal Operations as an Enabling Technology for Equivalent Visual Operations

Synthetic Vision (SV) may serve as a revolutionary crew/vehicle interface enabling technology to meet the challenges of the Next Generation Air Transportation System Equivalent Visual Operations (EVO) concept - that is, the ability to achieve or even improve on the safety of Visual Flight Rules (VFR) operations, maintain the operational tempos of VFR, and potentially retain VFR procedures independent of actual weather and visibility conditions. One significant challenge lies in the definition of required equipage on the aircraft and on the airport to enable the EVO concept objective. An experiment was conducted to evaluate the effects of the presence or absence of SV, the location (head-up or head-down) of this information during an instrument approach, and the type of airport lighting information on landing minima. Another key element of the testing entailed investigating the pilot's awareness and reaction to non-normal events (i.e., failure conditions) that were unexpectedly introduced into the experiment. These non-normals are critical determinants in the underlying safety of all-weather operations. This paper presents the experimental results specific to pilot response to non-normal events using head-up and head-down synthetic vision displays.

Documentary Reenactment and the Fantasmatic Subject

Reenactments, the more or less authentic re-creation of prior events, provided a staple element of documentary representation until they were slain by the “verite boys” of the 1960s (Robert Drew, Ricky Leacock, D. A. Pennebaker, David and Albert Maysles, Fred Wiseman, and others), who proclaimed everything except what took place in front of the camera without rehearsal or prompting to be a fabrication, inauthentic. Observational or direct cinema generated an honest record of what would have happened had the camera not been there or what does happen as a result of the camera recording people who know they are being filmed. Robert Flaherty’s Nanook of the North might be admired for the evidence it provides of Flaherty’s patience, exquisite eye, and apparent lack of preconceptions, but his entire salvage anthropology model of coaxing Allakariallak to do what “Nanook” would have done some thirty years earlier, without motorized vehicles, rifles, canned food, wood-frame homes, or filmmakers along for the ride, amounted to one colossal, unacknowledged reenactment and, therefore, fraud. Times have changed. Reenactments once again play a vital role in documentary, be they of the Solidarity movement that cannot be filmed in Far from Poland or of a murder for which radically disparate accounts exist in The Thin Blue Line, the schematic simulation of a harrowing escape in Little Dieter Needs to Fly or of events during the final days of Salvador Allende’s socialist government in Chile, Obstinate Memory. Apart from the occasional charges of deceit that surround the use of reenactments indistinguishable from actual footage of an historical event, reenactments are

Comparing perceptual judgment and subjective measures of spatial awareness

Spatial awareness is important in domains where safety hinges on human operators keeping track of the relative locations of objects in the environment. While a variety of subjective and judgment-based measures have been used to evaluate spatial awareness, none have probed all three of its levels: (1) identification of environmental objects, (2) their current locations relative to the operator, and (3) their relative positions over time. This work compares new judgment-based measures of spatial awareness that probe all three levels of spatial awareness to conventional subjective measures. In the evaluation of 14 configurations of Synthetic Vision Systems head down displays (seven terrain textures and two Geometric Fields of View (GFOVs)), 18 pilots made four types of judgments (relative angle, distance, height, and abeam time) regarding the location of terrain points displayed in 112 5-s, non-interactive simulations. They also provided subjective demand, awareness, clutter, SA-SWORD, and preferred GFOV measures. Correlation analyses revealed that displays that received higher awareness and SA-SWORD subjective ratings were associated with smaller errors in abeam time judgments and, for SA-SWORD, smaller errors in relative distance judgments. Thus SA-SWORD provides insight into level 2 spatial awareness and both SA-SWORD and awareness provide insight into level 3 spatial awareness. ANOVA and χ 2 analyses revealed comparable results between display configurations that produced the minimum error in judgments and those recommended by the awareness, SA-SWORD, and preferred GFOV measures.

Using Relative Position and Temporal Judgments to Identify Biases in Spatial Awareness for Synthetic Vision Systems

Synthetic vision systems (SVS) are cockpit displays that depict terrain ahead of ownship to prevent controlled flight into terrain. This work investigated how spatial biases manifest themselves in SVS displays. Eighteen pilots made spatial judgments (relative angle, distance, height, and abeam time) regarding the location of terrain points displayed in 112 5-sec videos of SVS displays. Judgment error characterized spatial biases related to between-map scale differences in geometric field of view, within-map differences in distance, within-map differences in orientation, the virtual space effect, the filled distance effect, and time. Recommendations for future experimentation and modeling are made.

Ecological Approach to Support Pilot Terrain Awareness After Total Engine Failure

Terrain awareness enhancing avionics, such as Synthetic Vision Systems and the Enhanced Ground Proximity Warning System, have been developed to reduce the number of controlled flight into terrain accidents. The protection these systems offer, however, is far from optimal. Synthetic Vision Systems only provide pilots with perceptual data, and leave all cognition and interpretation of data to the pilot. With Enhanced Ground Proximity Warning Systems the opposite is true. Here, pilots are only confronted with compelling advisories and commands without the underlying data and rationale. This paper presents a display system, the Emergency Landing Guidance System, that visualizes the functional of surrounding terrain, adopting an ecological interface design approach. The potential benefits of this approach are demonstrated with the case of locating and approaching a suitable landing area after sudden complete engine failure. To evaluate the effect of the new display on pilot terrain awareness, an experiment was conducted in a fixed-base flight simulator. Results show that the new display supports pilot terrain awareness much better than present terrain avoidance systems, i.e., pilots better understand the meaning of the terrain in relation to their goals and constraints. However, pilots also operated more closely to the limits of performance, thereby negatively affecting the metrics for safety.

Atonement and the Crime of Seeing: Patrick White's, Riders in the Chariot

Patrick White's ‘religious’ vision in ‘Riders in the Chariot’ is explored. Against readings from a Jewish perspective, a Christian perspective or a secular non-religious perspective, this article proposes an alternative. It suggests that White brings together a synthetic vision in which three religious traditions are affirmed as complimentary—with a new fourth, the artist as religious visionary. In doing so, White respects the deep differences between these traditions while also drawing on an analogical commonality: the redemptive value of suffering and the nature of an unfinished atonement within our lives.

Spatial Awareness in Synthetic Vision Systems: Using Spatial and Temporal Judgments to Evaluate Texture and Field of View

This work introduced judgment-based measures of spatial awareness and used them to evaluate terrain textures and fields of view (FOVs) in synthetic vision system (SVS) displays. SVSs are cockpit technologies that depict computer-generated views of terrain surrounding an aircraft. In the assessment of textures and FOVs for SVSs, no studies have directly measured the three levels of spatial awareness with respect to terrain: identification of terrain, its relative spatial location, and its relative temporal location. Eighteen pilots made four judgments (relative azimuth angle, distance, height, and abeam time) regarding the location of terrain points displayed in 112 noninteractive 5-s simulations of an SVS head-down display. There were two between-subject variables (texture order and FOV order) and five within-subject variables (texture, FOV, and the terrain point's relative azimuth angle, distance, and height). Texture produced significant main and interaction effects for the magnitude of error in the relative angle, distance, height, and abeam time judgments. FOV interaction effects were significant for the directional magnitude of error in the relative distance, height, and abeam time judgments. Spatial awareness was best facilitated by the elevation fishnet (EF), photo fishnet (PF), and photo elevation fishnet (PEF) textures. This study supports the recommendation that the EF, PF, and PEF textures be further evaluated in future SVS experiments. Additionally, the judgment-based spatial awareness measures used in this experiment could be used to evaluate other display parameters and depth cues in SVSs.

On Real-Time Synthetic Primate Vision

AbstractThe primate vision system exhibits numerous capabilities. Some important basic visual competencies include: 1) a consistent representation of visual space across eye movements; 2) egocentric spatial perception; 3) coordinated stereo fixation upon and pursuit of dynamic objects; and 4) attentional gaze deployment. We present a synthetic vision system that incorporates these competencies.We hypothesize that similarities between the underlying synthetic system model and that of the primate vision system elicit accordingly similar gaze behaviors. Psychophysical trials were conducted to record human gaze behavior when free-viewing a reproducible, dynamic, 3D scene. Identical trials were conducted with the synthetic system. A statistical comparison of synthetic and human gaze behavior has shown that the two are remarkably similar.

Piaget No `Remedy' for Kuhn, But the Two Should be Read Together

In arguing that the philosophical works of Jean Piaget could be used as a `remedy' for the flaws in those of Thomas Kuhn, Tsou overlooked some crucial aspects of the problem: the early history between them, the biological foundation supporting Piaget's method, and a preexisting suggestion regarding the intended future extension of his work. There was also no mention of the existence of a `lost' manuscript by Kuhn, which supposedly presents the mature articulation of his theory. This comment therefore proposes some `friendly amendments' to Tsou's exposition, with a view to helping achieve his synthetic vision once the `lost' work has finally been published. Yet the basic message, in anticipation of this future endeavor, is also exceedingly simple: the implicit direction of Piaget's (and Kuhn's) epistemological constructivism can be characterized as evolutionary-developmental `progress from,' rather than vitalist-teleological `progress toward.'