Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy in Helicopter Flight Simulation

Abstract

The advent of Virtual Reality (VR) in flight simulation promises to provide a cost-effective alternative for flight crew training compared to conventional flight simulation methods. However, it has been noted that the use of VR in flight simulation can lead to a greater incidence of cybersickness, which could jeopardize the effectiveness of flight training in VR. To optimally leverage the benefits that VR in flight simulation can bring, it is critical that this higher likelihood of experiencing cybersickness is countered. Even though a variety of theories for the causes for cybersickness in VR have been formulated, one of the most widely-accepted theories hinges on the principle that the sensory conflict between the visual sensory inputs from the virtual environment and the motion that is sensed by the vestibular system can result in cybersickness. Minimizing this sensory conflict can therefore be a strategy to mitigate cybersickness. The use of Mixed Reality (MR), in which the virtual environment is visually blended with the actual environment, could potentially be used for this strategy, based on the idea that it provides a visual reference of the actual environment that corresponds with the motion that is sensed, thereby reducing the sensory conflict and, correspondingly, cybersickness.The objective of this research is to investigate the effectiveness of MR, as an alternative for VR, for the mitigation of cybersickness in helicopter flight simulation. Since the idea of using MR as a cybersickness mitigation strategy is rooted in the idea of reducing the mismatch between visual and vestibular sensory inputs, the effectiveness of MR in combination with simulator motion is investigated as well. Arguably, MR could deteriorate immersion and reduce simulation fidelity, which may hamper the ability of the pilot to adequately fly in the virtual environment. Based on this premise, it is expected that a sweet spot exists where cybersickness is reduced, while fidelity remains sufficient to perform the flying task satisfactorily. In addition to evaluating the effectiveness for cybersickness mitigation, the impact of MR on pilot performance is also investigated.A human-in-the-loop experiment was performed that featured a total of four conditions, designed to assess the impact of both MR and motion on the cybersickness development and pilot performance. The experiment was performed in a simulated AgustaWestland AW139 helicopter on a Motion Systems’ PS-6TM-150 motion platform (6DoF), combined with a Varjo XR-3 visual device. In the experiment, Royal Netherlands Air Force helicopter pilots (n=4) were instructed to fly a series of maneuvers from the ADS-33 helicopter handling qualities guidelines. The Pirouette task in ADS-33 is the main focus for the results analysis because it is expected that near ground dynamic maneuvering affects cybersickness more severely compared to more stable and high altitude performed tasks.The cybersickness is evaluated by means of MIsery SCale (MISC) scores that were reported by the participants after the maneuver, and through a qualitative assessment triggered by VR comfort criteria. Pilot performance is assessed by examining the flight trajectories, helicopter inputs, and relevant helicopter outputs.The experiment campaign was completed in the last quarter of 2022, for which the results analysis is still ongoing. Preliminary results suggest that MR in the absence of simulator motion can have a beneficial effect on the development of cybersickness, as lower MISC scores were observed compared to the other conditions. However, the addition of motion with MR seems to have the opposite effect on cybersickness. Key in this result is that participants mentioned that being able to observe the simulator motion washout – the sensory-subliminal movement which allows the simulator to return to the neutral position – was an important sickening factor. The paper features a more in-depth analysis of the MISC scores and post-exposure sickness questionnaire results to further support the findings regarding the effect of MR and motion on cybersickness.While the analysis of the pilot performance results requires additional effort, current analysis of the flight trajectories and helicopter outputs suggests that MR may have a detrimental effect on pilot performance. The analysis of the pilots’ control input, however, still has to be completed. Next to an analysis of the flight trajectories and the helicopter outputs, the paper’s results also includes an analysis of the pilots’ control input – both in the time and frequency domains – to allow for a more detailed analysis on how the mitigating measures affect the individuals’ helicopter handling qualities and to which extent this influences the experienced cybersickness.