An Intelligent Real-Time Object Detection System on Drones

Abstract

Drones have been widely used in everyday life and they can help deal with various tasks, including photography, searching, and surveillance. Nonetheless, it is difficult for drones to perform customized online real-time object detection. In this study, we propose an intelligent real-time object detection system for drones. It is composed of an FPGA and a drone. A neural-network (NN) engine is designed on the FPGA for NN model acceleration. The FPGA receives activation data from an NN model, which are assembled into the data stream. Multiple fetch and jump pointers catch required activation values from the data stream, which are then filtered and sent to each thread independently. To accelerate processing speed, multiple processing elements (PEs) deal with tasks in parallel by using multiple weights and threads. The image data are transferred from the drone host to the FPGA, which are tackled with high speed by the NN engine. The NN engine results are returned to the host, which is used to adjust the flying route accordingly. Experimental results reveal that our proposed FPGA design well utilizes FPGA computing resources with 81.56% DSP and 72.80% LUT utilization rates, respectively. By using the Yolov3-tiny model for fast object detection, our system can detect objects at the speed of 8 frames per second and achieves a much lower power consumption compared to state-of-the-art methods. More importantly, the intelligent object detection techniques provide more pixels for the target of interest and they can increase the detection confidence score from 0.74 to 0.90 and from 0.70 to 0.84 for persons and cars, respectively.