Lightweight unmanned aerial vehicle video object detection based on spatial‐temporal correlation

Abstract

SummaryIntelligent unmanned aerial vehicles (UAVs) are drawing more and more attention from industry to academia. UAV navigation plays an important role in the cooperative scenario where multiple UAVs are deployed, while image data that capture the information of the UAV area are often used as input for UAV navigation. Deep learning is a common and powerful technique for UAV image processing, but a complex model generated by deep learning technique is hardly suitable for the limited computing capacity of edge computing devices such as UAVs. Therefore, this paper designs an efficient deep learning model on UAVs to fit the restriction of low computational powers and low power consumption. Traditional UAV object detection methods mostly use static images as the basis for object recognition, or collect images for offline detection. Our method combines the existing fast single‐frame detection methods with the spatial‐temporal relationship of video sequences, to build an efficient end‐to‐end model. In addition, the convolutional LSTM module is used to propagate the temporal context of the video frame sequences. Based on the temporal context, we propose a module for calculating spatial correlation. At the same time, we establish our experimental dataset in our real application and conduct the experiment, which shows that the proposed method reduces the size of models and meanwhile maintains the detection rate. Compared with the existing static images approaches, our method is faster and more accurate. Inference speeds of nearly 20fps can be achieved while performing real‐time tasks.