Manifold Siamese Network: A Novel Visual Tracking ConvNet for Autonomous Vehicles

Abstract

Visual tracking is a vital component of autonomous driving perception system. Siamese networks have achieved great success in both accuracy and speed for visual tracking tasks. These Siamese trackers share a similar framework in which each tracker consists of two network branches for exploring semantic information. However, the performance of Siamese trackers is limited by an insufficient semantic template and an unsatisfactory updating strategy. To tackle these problems, we propose a manifold Siamese network for visual tracking that can simultaneously utilize semantic and geometric information. A manifold sample pool is constructed to exploit the manifold structure of image object sequences. This sample pool is dynamically learned via a fast Gaussian mixture model (GMM). After obtaining a manifold sample template, we design a deep architecture based on a correlation filter (CF) network and append a novel manifold feature branch. The network remains fully convolutional and can train a template to discriminate exemplar image and arbitrarily size search image. Then, a triplet occlusion score function cooperates with an effective update method that is established to prevent model drift. Extensive experiments show that the proposed tracking algorithm performs favorably compared with the state-of-the-art methods on three standard benchmark datasets at a high framerate, which is very suitable for autonomous driving.