Obstacle visual sensing based on deep learning for low-altitude small unmanned aerial vehicles

Abstract

An obstacle real-time sensing method, which is based on deep learning and target tracking method and integrated with monocular vision and binocular vision, is proposed for unmanned aerial vehicles (UAVs) in this paper. Firstly, it uses the deep learning method to detect and recognize the first-frame figure collected by cameras. Then, it uses the target tracking algorithm to track the detection results for the first-frame figure in real time to improve the real-time performance of the detection system. Meanwhile, it uses the binocular vision technology to execute the three-dimensional reconstruction for the current frame of the entire figure to obtain the environmental spatial information. Subsequently, combined with the points clustering strategy and the information fusion method, it can resolve the types, spatial locations, and outlines of obstacles. Finally, to verify the proposed method, we developed a physical prototype, and the results showed that the real-time sensing for obstacles can be realized under the condition that UAVs are equipped with one binocular camera.