Measurement and reduction of system latency in see-through helmet mounted display (HMD) systems

Abstract

Future military aviation platforms such as the proposed Joint Strike Fighter F-35 will integrate helmet mounted displays (HMDs) with the avionics and weapon systems to the degree that the HMDs will become the aircraft's primary display system. In turn, training of pilot flight skills using HMDs will be essential in future training systems. In order to train these skills using simulation based training, improvements must be made in the integration of HMDs with out-thewindow (OTW) simulations. Currently, problems such as latency contribute to the onset of simulator sickness and provide distractions during training with HMD simulator systems that degrade the training experience. Previous research has used Kalman predictive filters as a means of mitigating the system latency present in these systems. While this approach has yielded some success, more work is needed to develop innovative and improved strategies that reduce system latency as well as to include data collected from the user perspective as a measured variable during test and evaluation of latency reduction strategies. The purpose of this paper is twofold. First, the paper describes a new method to measure and assess system latency from the user perspective. Second, the paper describes use of the testbed to examine the efficacy of an innovative strategy that combines a customized Kalman filter with a neural network approach to mitigate system latency. Results indicate that the combined approach reduced system latency significantly when compared to baseline data and the traditional Kalman filter. Reduced latency errors should mitigate the onset of simulator sickness and ease simulator sickness symptomology. Implications for training systems will be discussed.