Aircraft tracking in aerial videos based on fused RetinaNet and low‐score detection classification

Abstract

Aircraft tracking in aerial videos is a challenge due to the small size and appearance affinity with other targets. Recently, the tracking-by-detection trackers, such as simple online and realtime tracking (SORT), attracted the attention of many researchers due to the great evolution in deep learning-based object detectors. SORT depends on motion cues in solving the data association problem by matching high-score detection bounding boxes to tracklets. However, motion cues are insufficient to track occluded or similar aircraft efficiently. In addition, most occluded aircrafts have a low-confidence percentage. The main objective of this paper is to propose an F-SORT tracker, an enhanced SORT tracker with the following improvements. Firstly, fused RetinaNet for small aerial targets is used as a detection framework to boost the detectability of SORT. Then, aircraft appearance features are extracted from the shallowest prediction layer and utilized as additional cues to improve the data association algorithm. Finally, the extracted features distinguish aircraft online from backgrounds via a K-nearest neighbour classifier. Experiments on an airport dataset show that F-SORT can significantly improve aircraft tracking. F-SORT outperforms other state-of-the-art trackers, achieving 72.75%, 59.63%, and 82.89% on MOTA, HOTA, and IDF1. In addition, F-SORT classifying algorithm boosts other trackers when applied.