Attention shake siamese network with auxiliary relocation branch for visual object tracking

Abstract

Siamese network is highly regarded in the visual object tracking filed because of its unique advantages of pairwise input and pairwise training. It can measure the similarity between two image patches, which coincides with the principle of the matching-based tracking algorithm. In this paper, a variant Siamese network based tracker is proposed to introduce attention module into traditional Siamese network, and relocate the object with some auxiliary relocation methods, when the proposed tracker runs under an untrusted state. Firstly, a novel attention shake layer is proposed to replace the max pooling layer in Siamese network. This layer could introduce and train two different kinds of attention modules at the same time, which means the proposed attention shake layer could also help to improve the expression power of Siamese network without increasing the depth of the network. Secondly, an auxiliary relocation branch is proposed to assist in object relocation and tracking. According to the prior assumptions of visual object tracking, some weights are involved in the auxiliary relocation branch, such as structure similarity weight, motion similarity weight, motion smoothness weight and object saliency weight. Thirdly, a novel response map based switch function is proposed to monitor the tracking process and control the effect of auxiliary relocation branch. Furthermore, in order to discuss the effect of pooling layer in Siamese network, 9 pooling and attention architectures are proposed and discussed in this paper. Some empirical results are shown in the experiment part. Comparing with the state-of-the-art trackers, the proposed tracker could achieve comparable performance in multiple benchmarks.