Using deep learning in an embedded system for real-time target detection based on images from an unmanned aerial vehicle: vehicle detection as a case study

Abstract

ABSTRACT For a majority of remote sensing applications of unmanned aerial vehicles (UAVs), the data need to be downloaded to ground devices for processing, but this procedure cannot satisfy the demands of real-time target detection. Our objective in this study is to develop a real-time system based on an embedded technology for image acquisition, target detection, the transmission and display of the results, and user interaction while providing support for the interactions between multiple UAVs and users. This work is divided into three parts: (1) We design the technical procedure and the framework for the implementation of a real-time target detection system according to application requirements. (2) We develop an efficient and reliable data transmission module to realize real-time cross-platform communication between airborne embedded devices and ground-side servers. (3) We optimize the YOLOv4 algorithm by using the K-Means algorithm and TensorRT inference to improve the accuracy and speed of the NVIDIA Jetson TX2. In experiments involving static detection, it had an overall confidence of 89.6% and a rate of missed detection of 3.8%; in experiments involving dynamic detection, it had an overall confidence and a rate of missed detection of 88.2% and 4.6%, respectively.