End-Side Gesture Recognition Method for UAV Control

Abstract

Traditional heavy and complex control equipment such as remote controllers (RCs) and ground stations (GSs) are not able to meet the fast and flexible control requirements of unmanned aerial vehicles (UAVs) in complex environments. This article thus proposes a gesture recognition method for the control of UAVs. To achieve the goal of gestural manipulation of UAVs, this method can accurately recognize a variety of gestures and convert the gestures into corresponding UAV control commands. First, the wireless data glove incorporates flex sensors, and inertial sensors are used to gather gesture datasets. The trained neural network model is then deployed on the data glove based on an STM32 microcontroller for real-time gesture recognition, in which the backpropagation (BP) network is used for static gesture recognition and the bidirectional gated recurrent unit (Bi-GRU) network is used for dynamic gesture recognition. Finally, the gesture is converted into the control command and sent to the aircraft terminal to control the UAV. Using the UAV simulation test on the simulation platform, the recognition accuracy of the basic UAV control commands corresponding to ten static gestures is found to be 100%, and the mean recognition accuracy of the UAV mode-switching commands corresponding to five dynamic gestures is 98.4% indicating that the gesture recognition effect of the system is perfect. Mission testing performed in the scene constructed in the simulation environment demonstrates that the UAV can respond rapidly to gestures, and the method proposed in this article can achieve real-time and stable control of the UAV on the end side.