Robust object tracking via ensembling semantic‐aware network and redetection

Abstract

AbstractMost Siamese‐based trackers use classification and regression to determine the target bounding box, which can be formulated as a linear matching process of the template and search region. However, this only takes into account the similarity of features while ignoring the semantic object information, resulting in some cases in which the regression box with the highest classification score is not accurate. To address the lack of semantic information, an object tracking approach based on an ensemble semantic‐aware network and redetection (ESART) is proposed. Furthermore, a DarkNet53 network with transfer learning is used as our semantic‐aware model to adapt the detection task for extracting semantic information. In addition, a semantic tag redetection method to re‐evaluate the bounding box and overcome inaccurate scaling issues is proposed. Extensive experiments based on OTB2015, UAV123, UAV20L, and GOT‐10k show that our tracker is superior to other state‐of‐the‐art trackers. It is noteworthy that our semantic‐aware ensemble method can be embedded into any tracker for classification and regression task.