Evaluating Pilot’s Perceived Workload on Interacting with Augmented Reality Device in Flight Operations

Abstract

Augmented Reality (AR) is a tool which can be used to improve human-computer interaction in flight operations. The application of AR can facilitate pilots integrating the information from interfaces in the flight deck to analyze various sources of messages simultaneously. There are seventeen subjects aged from 23 to 53 (M = 29.82, SD = 8.93) who have participated in this experiment. Their flight experience ranged from zero flight hours to 3000 flight hours (M = 605.00, SD = 1051.04). Two types of HCI AR design (gesture or voice control checklist) have been compared with traditional paper checklist. The results show that AR gesture control induced the highest perceived workload compared with AR voice checklist and traditional paper checklist. There are lots of complicated cognitive processes and physical movements involved in the AR gesture checklist that induced the highest level of effort and frustration based on NASA-TLX. The AR checklist application has relied on the use of the default HoloLens interactions including cursor movement linked with head movements, Air Tap gesture and Microsoft voice recognition system. The current technological features embedded in the HoloLens device are not certified to be used in the cockpit yet. The improvement in the types of interaction and displays with AR devices could lead to changes in pilot’s perceived workload while interacting with an innovative device. This research demonstrated that AR integrated with voice command has potential of significant benefits to be applied in the flight deck for future flight operation.