Multi-modal human-computer interaction system in cockpit

Abstract

In order to explore new human-machine interaction methods, a set of multi-modal human-machine interaction coordinated control system is proposed, which realizes the basic flight control based on the change of the pilot’s field of view, touch control, voice control and other information obtained from multi-mode coordinated control. This system introduces a new type of human-computer interaction into the cockpit application on the basis of the human-computer interaction interface for flight driving, and has carried out research on multi-mode collaborative control system, mainly including eye movement interaction, touch interaction gesture Interaction and voice interaction. Finally, this project formed a multi-mode cooperative control software and hardware system.