Helmet Mounted Display System Research Articles

Measuring cognitive load of digital interface combining event-related potential and BubbleView

Helmet mounted display systems (HMDs) are high-performance display devices for modern aircraft. We propose a novel method combining event-related potentials (ERPs) and BubbleView to measure cognitive load under different HMD interfaces. The distribution of the subjects’ attention resources is reflected by analyzing the BubbleView, and the input of the subjects’ attention resources on the interface is reflected by analyzing the ERP’s P3b and P2 components. The results showed that the HMD interface with more symmetry and a simple layout had less cognitive load, and subjects paid more attention to the upper portion of the interface. Combining the experimental data of ERP and BubbleView, we can obtain a more comprehensive, objective, and reliable HMD interface evaluation result. This approach has significant implications for the design of digital interfaces and can be utilized for the iterative evaluation of HMD interfaces.

Helmet-Mounted Display System Based on IoT

Helmet-Mounted Display System Based on IoT

Eye Gaze Estimation Based on Stacked Hourglass Neural Network for Aircraft Helmet Aiming

Helmet-mounted display (HMD) systems allow aircraft pilots to aim at targets by using head postures. However, the direct use of helmet orientation to indicate the aiming direction ignores human eye movements, which are more flexible and efficient for interaction. Since the opaqueness of the helmet goggle blocks the sight of external cameras to capture facial or eye images of the pilots, and traditional eye feature extraction methods may fail when encountering conditions such as poor lighting, occlusion, and shaking, which are common on fighter aircrafts. In this work, an eye gaze based aiming solution that adapts to pilots wearing HMDs is proposed, and a deep learning-based method is proposed to extract eye features robustly. The prototype experiments demonstrate the ability to pick and aim at targets in real-time (60FPS) and are capable to accurately locate the target markers on a screen with an average error of fewer than 2 degrees. Conclusively, the proposed method performs the tasks of eye feature extraction on real-person imagery and the estimation of the 3D aiming direction for users with helmets, displaying competitive results with similar research.

Hybrid Helmet Attitude Tracking System using Adaptive Complementary Filter

This paper describes the design and performance analysis of HTS (Helmet attitude Tracking System), providing coincident LOS (Line of Sight) between aircraft and HMD (Helmet Mounted Display) which displays flight and mission information on Pilot helmet. The functionality and performance of the HMD system depends on the performance of HTS (Helmet Tracking System). The goal of HTS system design is to meet real-time performance by synchronizing the latency of non-periodic sensory systems and high accurate tracking performance. To prove the performance of the proposed approach using ACF (Adaptive Complementary Filter), a robust hybrid attitude tracking scheme with a fusion of optical sensor and inertial sensor, it took an experiment through an implemented test-bed. The experimental results show accurate and reliable tracking performance despite external errors.

Application of Multi-Criteria Optimization to Choose Variants of Hardware Architecture for the SWPL-1 Helmet-Mounted Display System

Application of Multi-Criteria Optimization to Choose Variants of Hardware Architecture for the SWPL-1 Helmet-Mounted Display System

Modelling of Physical Phenomena in the Vertical Velocity Measuring Circuit for Process of Designing the SWPL-1 Helmet-Mounted Display System

Modelling of Physical Phenomena in the Vertical Velocity Measuring Circuit for Process of Designing the SWPL-1 Helmet-Mounted Display System

Possibility of upgrading fighter aircraft MiG-29 in the field of helmet-mounted display systems

Possibility of upgrading fighter aircraft MiG-29 in the field of helmet-mounted display systems

Daylight luminance requirements for full-color, see-through, helmet-mounted display systems

When color is implemented in helmet-mounted displays (HMDs) that are eyes-out, see-through displays, visual perception issues become an increased concern. A major confound with HMDs is their inherent see-through (transparent) property. The result is color in the displayed image that combines with color from the outside (or in-cockpit) world, producing an image with additive color. As luminance of the HMD imagery is reduced, the color separation between the HMD imagery and the background is also reduced. It is because of this additive effect that luminance contrast is so vitally important in developing HMD standards for color symbology. As a result, this paper identifies luminance requirements for full-color HMDs based upon two lines of investigation. The first is based on a study of white symbology against natural static backgrounds, where the quality of symbology was judged to be a function of not only the background luminance but also of the background complexity as well. The second is based on an evaluation of the complexity inherent in natural backgrounds and from this investigation, a predictive curve was found that describes the complexity of natural backgrounds as a function of ambient luminance.

Application of large head-box aircraft transparency distortion measurement and compensation for improved cueing in helmet-mounted displays

See-through distortion can occur when a pilot looks through a curved aircraft transparency because of a weak lens effect. A helmet-mounted display (HMD) system must either compensate for this distortion or incur a distortion error in its total error budget. We describe a method for characterization of and compensation for aircraft transparency distortion, allowing for increased targeting accuracy and reduced binocular vergence errors from any position within a large head box. We present techniques for compensating for transparency distortion for the entire HMD head box and implications observed based on applying these techniques for A-10 and F-16 aircraft transparencies.

Developing an optical channel for a built-in motorcycle-helmet-mounted display system

This paper discusses the possibility of creating a collimator-type helmet-mounted display system based on a scanning microprojector. It is shown that there is a connection between the type of projector and the composition of the optical display system, as well as the functional purpose of its separate modules. Methods are described for configuring a decentered optical system, and an example is presented with the results of the calculation.

실시간 헬멧자세 추적시스템의 설계 및 구현

This paper describes the design and implementation scheme of HTS(Helmet Tracking System) providing coincident LOS(Line of Sight) between aircraft and HMD(Helmet Mounted Display) which displays flight and mission information on Pilot helmet. The functionality and performance of HMD system depends on the performance of helmet tracking system. The target of HTS system design is to meet real-time performance and reliability by predicting non-periodic latency and high accuracy performance. To prove an availability of a proposed approach, a robust hybrid scheme with a fusion optical and inertial tracking system are tested through a implemented test-bed. Experimental results show real-time and reliable tracking control in spite of external errors.

Flight Parameter Display System For The Mi-17 Helicopter – Test Results

AbstractThis article presents flight tests of a newly developed Helmet-Mounted Flight Parameters Display System (SWPL). Presented system is designed to illustrate full piloting and navigational information, warning about emergency on the helicopter’s board and signaling on-board helicopter systems failures on the translucent display. The system is designed to work in day and night time conditions. In night time conditions, the system cooperates with night vision goggles applicable in Polish Air Force. The article presents the main components of the tested system, along with their purpose and function. It also describes in detail the methods of display implemented in the system and the amount of displayed information. The article discusses the required range and the actual course of the flight tests of that system. Tests were conducted on a Mi-17-1V helicopter. The results of the flight tests of the Helmet-Mounted Display System are given in conclusion, in particular regarding meeting the tactical and technical requirements of the system.

An approach for calculating the optical structure based on ybar-y diagram

The first-order quantity of the optical structure can be obtained using the ybar-y diagram and this diagram has a control point quantity. Based on the concept and quantity of the ybar-y diagram, this paper establishes a mathematical model to calculate an optical system’s first-order structure. The aberration is induced by the deflection of the transmission rays. If a first-order structure has a minimal total deflection angle of the first paraxial ray and the second paraxial ray, the higher order aberration will have a small value and the calculated first-order optical structure will be the best. According to the mathematical model, the issue of optical structure calculation is converted to a numerical optimization problem. And the objective function, i.e. the sum of the first and second paraxial ray’s deflection angles, is constructed. After comparison among etween many kinds of numerical optimization algorithms, the particle swarm optimization algorithm is used to solve the problem. Then a calculation program containing graphical user interface (GUI) is developed to calculate the first-order structure quickly and efficiently. The basic design parameters of the optical system are imported into the GUI after some treatments, then the resulting first-order structure is obtained after some clicks of the mouse. The resulting structure is thereafter converted to a practical lens system by the use of commercially available optical design software such as ZEMAX. After a lens optimization process, an actual optical system is accomplished. According to the method proposed in this paper and by the use of the calculation program, a 13 megapixel mobile phone camera lens is designed first. The F# of this lens system is 2.3 and the full field of viewing angle is 70 degree. The system has a total length of 4.5 mm and a distortion of 1.2%. Only four aspheric lenses are used and the other optical performance meets the design requirements as well. In addition, an eyepiece of helmet-mounted display system is designed, in which only two lenses are used and a visual field angle of 90 degree is achieved. The entrance pupil is 5 mm width and the image diagonal length is 65 mm. This system has a total length less than 45 mm and eye relief greater than 12 mm. Other performance of the eyepiece can also meet the requirements. These designs of the two optical systems demonstrate that the proposed method is reliable in calculating the optical structure of the optical system.

自由曲面机载头盔显示器光学系统设计

In order to resolve the contradictions between large exit pupil, wide field of view (FOV) and lightweight, small size, a helmet mounted displays (HMDs) system is designed, using freeform surface described by radial basis function. The principle that radial basis function describes freeform surface is introduced, and the approaches for aberration correction are analyzed. A new way is tried to determine a starting point of optimization quickly while designing. At last, the imaging quality of the optical system is analyzed. A HMD with 4532, 15 mm pupil, 50 mm eye relief is designed. The modulation transfer function (MTF) value is more than 0.6 within the entire FOV at the Nyquist frequency. The magnitude of maximum distortion occurs at (-22.5, 16) in the field and it is measured in simulation to be -1.54%. The size of the system is 56 mm128 mm and the weight is 136 g. The design results show that the optical system of HMDs has low aberration, and it can provide user clear symbol or video images. This HMDs system, which has good image quality, small volume, and light weight, can be applied to the next generation avionic HMD technology.

Improving visual-conformal displays for helicopter guidance

A new layout of visual-conformal display for helicopter guidance is presented. It offers a clearer graphics of a helmet-mounted display system that can enhance guidance technology to improve safety of helicopter missions.

Design of an off-axis helmet-mounted display with freeform surface described by radial basis functions

An off-axis optical system with a freeform surface described by Gaussian radial basis functions for a see-through wide field of view helmet-mounted display (HMD) is presented. By using a freeform surface, an off-axis see-through HMD system with one tilted freeform combiner and four relay optical lenses is designed. An off-axis see-through wide field of view HMD system with a 100mm eye relief, 15mm pupil, 45°×32° FOV, and 60° combiner tilt angle is achieved. For the purpose of comparison, two other off-axis see-through HMDs which have the same specifications, but different surface type of combiner and different use count of relay lenses are designed too. One of the two contributions in this paper is the application of radial basis functions to describe optical freeform surface in a wide field of view off-axis helmet-mounted displays, and the other is a way used to determine a starting point of optimization quickly while designing.

Mitigation of System Latency in Next Generation Helmet Mounted Display Systems (Nghmds)

The ability of helmet mounted display (HMD) systems to increase effectiveness in operational aircraft has been well documented over the last several years. Now that advanced aircraft are committed to using HMDs in combat operations, issues associated with their use in simulators must be addressed. A major factor associated with the onset of simulator sickness is system latency. Simulator sickness can be a significant distraction during training and may result in ineffective training, negative training, reduced user acceptance, and a reduction in simulator usage. Innovative solutions to address system latency, as well as many other issues, must be developed so that training can be optimized. The current project developed a number of innovative strategies that effectively mitigate both system latency and image alignment error. The strategies developed include: 1) a customized Kalman predictive filter, 2) a learning neural network, and 3) “Warper Board” technology. These strategies operate independently yet in concert to continually sample, compare, and adjust their outputs to produce the most accurate prediction of future image placement possible using current available head movement and position data. Experimental results indicate that effective system latency was reduced up to 90% from the baseline system state. Implications for training systems and improved training will be discussed.

Poster 174: Old Age Reduces the Ability to Reorient Locomotor Trajectories Based on Visual Information

Objective: To investigate the influence of changing optic flows (OF) on the control of locomotor steering in young and older adults. Design: Quasi-experimental, mixed model design. Setting: Virtual reality (VR) gait laboratory, rehabilitation hospital. Participants: 20 young (age, 18–29y) and 20 older adults (age, 60–85y). Interventions: Participants were exposed to translational and rotational OFs viewed as a virtual room through a helmet-mounted display system during walking. The focus of expansion (FOE) was randomly translated or rotated to different locations: 40° right or left, or neutral (0°).

Hyperstereopsis Associated with Helmet-Mounted Sighting and Display Systems for Helicopter Pilots

Hyperstereopsis Associated with Helmet-Mounted Sighting and Display Systems for Helicopter Pilots

Helmet-mounted display image quality evaluation system

Helmet-mounted displays (HMDs) provide essential pilotage and fire control imagery information for pilots. To maintain system integrity and readiness, there is a need to develop an image quality evaluation system for HMDs. In earlier work, a framework was proposed for an HMD system called the integrated helmet and display sighting system (IHADSS), used with the U.S. Army's Apache helicopter. This paper describes prototype development and interface design and summarizes bench test findings using three IHADSS helmet display units (HDUs). The prototype consists of hardware (cameras, sensors, image capture/data acquisition cards, battery pack, HDU holder, moveable rack and handle, and computer) and software algorithms for image capture and analysis. Two cameras with different-size apertures are mounted in parallel on a rack facing an HDU holder. A handle allows users to position the HDU in front of the two cameras. The HMD test pattern is then captured. Sensors detect the position of the holder and whether the HDU is angled correctly in relation to the camera. Algorithms detect HDU features captured by the two cameras, including focus, orientation, displacement, field-of-view, and number of grayshades. Bench testing of three field-quality HDUs indicates that the image analysis algorithms are robust and able to detect the desired image features. Suggested future directions include development of a learning algorithm to automatically develop or revise feature specifications as the number of inspection samples increases.