SiamSMDFFF: Siamese network tracker based on shallow-middle-deep three-level feature fusion and clustering-based adaptive rectangular window filtering

Abstract

With the development of machine vision, target trackers based on Siamese networks have demonstrated excellent performance in balancing speed and accuracy compared to traditional methods. Traditional trackers usually use only the deep features extracted from the last convolutional neural network (CNN) layer, which contain semantic information, to complete the similarity matching. In addition, a cosine window is usually used to filter the score map, which leads to the limited suppression effect of the tracker on the interference of the objects similar to the target and the poor robustness of the tracker. In this paper, a Siamese network tracker named SiamSMDFFF, which combines shallow-middle-deep feature fusion with a clustering-based adaptive rectangular window filter, is proposed. SiamSMDFFF uses the features to fuse at the feature level to obtain the complementary feature maps and then uses the score maps calculated from the complementary feature maps via correlation to fuse at the score level and obtain the final score maps. Then, the peak points in the score map are used as the initial clustering centers to complete the clustering, and the distance between the clustering center and the farthest clustering point is calculated. Finally, the distance is used to control the change in the size of the rectangular window to filter the score map in order to overcome the negative impact of the similar targets interfering in the tracking process and improve the robustness of the tracker. The experimental results demonstrate that SiamSMDFFF is significantly improved in several aspects compared to the conventional tracker.