Natural UAV tele-operation for agricultural application by using Kinect sensor

Abstract

Remote-controlled (RC) unmanned aerial vehicles (UAVs) have been extensively applied in agricultural areas, such as remote sensing, precise spraying pesticides for crop protection, agricultural situation inspection and so on, but these telemanipulated UAVs systems are operated entirely by a ground-based pilot with a need of eyes focus on the remote site UAV flight. The key issue existed in agricultural UAV teleoperation area is a longtime training needed. In this paper, a novel natural UAV teleoperation control system in agricultural application was proposed. In UAV teleoperation scenario, human operator gestures measured by using Kinect sensor can be used as control commands for UAV flight. Moreover, some UAV teleoperation control commands related to human hand gestures were defined, which is similar to the radio gymnastic exercises in China. Therefore, gesture recognition-based UAV teleoperation control is easy to learn and master. In addition, a new real time human hand gesture recognition algorithm was proposed. The stability of UAV flight dynamic of roll, pitch and yaw as well as attitude control were verified with the experiments based on the proposed method. At last, the usability and effectiveness of the proposed method has been verified by the experimental results. Keywords: gesture recognition, human pose estimation, depth image, skeleton tracking, teleoperation, unmanned aerial vehicle (UAV) DOI: 10.25165/j.ijabe.20181104.4096 Citation: Yin X C, Lan Y B, Wen S, Zhang J T, Wu S F. Natural UAV tele-operation for agricultural application by using Kinect sensor. Int J Agric & Biol Eng, 2018; 11(4): 173-178.