Visual Tracking Benchmark Research Articles

High-speed tracking with multi-templates correlation filters

Correlation filter (CF)-based trackers have demonstrated impressive performance on visual tracking benchmark datasets with high frame rates. A lot of research focuses on extracting strong features to learn a robust model of the object’s appearance. However, the circularly shifted samples used in traditional CF training come from one single image patch and have limited information about the target. Inspired by various data augmentation methods in deep learning, we propose a general multi-templates correlation filter-based tracker that expands the CF tracker with multihandcrafted training samples. We reformulate the original optimization problem and provide a closed-form solution to maintain the high-speed calculation. Several methods to generate the additional samples are evaluated. Comprehensive experiments illustrate that our framework obviously improves the tracking performance and is general enough to be incorporated with classical CF trackers.

Siamese anchor-free object tracking with multiscale spatial attentions

Recently, object trackers based on Siamese networks have attracted considerable attentions due to their remarkable tracking performance and widespread application. Especially, the anchor-based methods exploit the region proposal subnetwork to get accurate prediction of a target and make great performance improvement. However, those trackers cannot capture the spatial information very well and the pre-defined anchors will hinder robustness. To solve these problems, we propose a Siamese-based anchor-free object tracking algorithm with multiscale spatial attentions in this paper. Firstly, we take ResNet-50 as the backbone network to generate multiscale features of both template patch and search regions. Secondly, we propose the spatial attention extraction (SAE) block to capture the spatial information among all positions in the template and search region feature maps. Thirdly, we put these features into the SAE block to get the multiscale spatial attentions. Finally, an anchor-free classification and regression subnetwork is used for predicting the location of the target. Unlike anchor-based methods, our tracker directly predicts the target position without predefined parameters. Extensive experiments with state-of-the-art trackers are carried out on four challenging visual object tracking benchmarks: OTB100, UAV123, VOT2016 and GOT-10k. Those experimental results confirm the effectiveness of our proposed tracker.

Intelligent Trigonometric Particle Filter for visual tracking

Visual tracking is one of the pre-eminent tasks in several computer vision applications. Particle filter (PF) is extensively used in visual tracking for intelligent surveillance system applications, hugely significant. But the re-sampling procedure of PF will result in sample impoverishment, which will affect the precision of tracking simultaneously. In this paper, a new tracking technique, called Trigonometric Particle Filter (TPF), based on PF optimized by Sine Cosine Algorithm (SCA), which contains trigonometric sine and cosine functions, is proposed. An enhanced method for improving the number of target particles used in a Sine Cosine Algorithm for trigonometric particle filter includes SCA ahead of the re-sampling step. This step ensures a more extensive particle set Achievement of the proposed TPF tracker is inspected and assessed on Visual Tracker Benchmark (VOT) databases. The proposed TPF tracker is compared with evolutionary-based methods like the Spider monkey optimization assisted PF (SMO-PF), Firefly algorithm-based PF (FAPF) method, Particle swarm optimization-based PF (PSO-PF) and Particle filter, recent four correlation filter-based trackers, and also with other ten state-of-the-art tracking methods. We demonstrate that visual tracking using TPF delivers additional consistent and proficient tracking outcomes than compared trackers.

SAFS: Object Tracking Algorithm Based on Self-Adaptive Feature Selection.

Object tracking is one of the most challenging problems in the field of computer vision. In challenging object tracking scenarios such as illumination variation, occlusion, motion blur and fast motion, existing algorithms can present decreased performances. To make better use of the various features of the image, we propose an object tracking method based on the self-adaptive feature selection (SAFS) algorithm, which can select the most distinguishable feature sub-template to guide the tracking task. The similarity of each feature sub-template can be calculated by the histogram of the features. Then, the distinguishability of the feature sub-template can be measured by their similarity matrix based on the maximum a posteriori (MAP). The selection task of the feature sub-template is transformed into the classification task between feature vectors by the above process and adopt modified Jeffreys’ entropy as the discriminant metric for classification, which can complete the update of the sub-template. Experiments with the eight video sequences in the Visual Tracker Benchmark dataset evaluate the comprehensive performance of SAFS and compare them with five baselines. Experimental results demonstrate that SAFS can overcome the difficulties caused by scene changes and achieve robust object tracking.

A New Approach to Enhanced Swarm Intelligence Applied to Video Target Tracking.

This work proposes a new approach to improve swarm intelligence algorithms for dynamic optimization problems by promoting a balance between the transfer of knowledge and the diversity of particles. The proposed method was designed to be applied to the problem of video tracking targets in environments with almost constant lighting. This approach also delimits the solution space for a more efficient search. A robust version to outliers of the double exponential smoothing (DES) model is used to predict the target position in the frame delimiting the solution space in a more promising region for target tracking. To assess the quality of the proposed approach, an appropriate tracker for a discrete solution space was implemented using the meta-heuristic Shuffled Frog Leaping Algorithm (SFLA) adapted to dynamic optimization problems, named the Dynamic Shuffled Frog Leaping Algorithm (DSFLA). The DSFLA was compared with other classic and current trackers whose algorithms are based on swarm intelligence. The trackers were compared in terms of the average processing time per frame and the area under curve of the success rate per Pascal metric. For the experiment, we used a random sample of videos obtained from the public Hanyang visual tracker benchmark. The experimental results suggest that the DSFLA has an efficient processing time and higher quality of tracking compared with the other competing trackers analyzed in this work. The success rate of the DSFLA tracker is about 7.2 to 76.6% higher on average when comparing the success rate of its competitors. The average processing time per frame is about at least 10% faster than competing trackers, except one that was about 26% faster than the DSFLA tracker. The results also show that the predictions of the robust DES model are quite accurate.

Deep Learning for Visual Tracking: A Comprehensive Survey

Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given the ill-posed nature of the problem and its popularity in a broad range of real-world scenarios, a number of large-scale benchmark datasets have been established, on which considerable methods have been developed and demonstrated with significant progress in recent years -- predominantly by recent deep learning (DL)-based methods. This survey aims to systematically investigate the current DL-based visual tracking methods, benchmark datasets, and evaluation metrics. It also extensively evaluates and analyzes the leading visual tracking methods. First, the fundamental characteristics, primary motivations, and contributions of DL-based methods are summarized from nine key aspects of: network architecture, network exploitation, network training for visual tracking, network objective, network output, exploitation of correlation filter advantages, aerial-view tracking, long-term tracking, and online tracking. Second, popular visual tracking benchmarks and their respective properties are compared, and their evaluation metrics are summarized. Third, the state-of-the-art DL-based methods are comprehensively examined on a set of well-established benchmarks of OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, and VisDrone2019. Finally, by conducting critical analyses of these state-of-the-art trackers quantitatively and qualitatively, their pros and cons under various common scenarios are investigated. It may serve as a gentle use guide for practitioners to weigh when and under what conditions to choose which method(s). It also facilitates a discussion on ongoing issues and sheds light on promising research directions.

MFCFSiam: A Correlation-Filter-Guided Siamese Network with Multifeature for Visual Tracking

With the development of deep learning, trackers based on convolutional neural networks (CNNs) have made significant achievements in visual tracking over the years. The fully connected Siamese network (SiamFC) is a typical representation of those trackers. SiamFC designs a two-branch architecture of a CNN and models’ visual tracking as a general similarity-learning problem. However, the feature maps it uses for visual tracking are only from the last layer of the CNN. Those features contain high-level semantic information but lack sufficiently detailed texture information. This means that the SiamFC tracker tends to drift when there are other same-category objects or when the contrast between the target and the background is very low. Focusing on addressing this problem, we design a novel tracking algorithm that combines a correlation filter tracker and the SiamFC tracker into one framework. In this framework, the correlation filter tracker can use the Histograms of Oriented Gradients (HOG) and color name (CN) features to guide the SiamFC tracker. This framework also contains an evaluation criterion which we design to evaluate the tracking result of the two trackers. If this criterion finds the SiamFC tracker fails in some cases, our framework will use the tracking result from the correlation filter tracker to correct the SiamFC. In this way, the defects of SiamFC’s high-level semantic features are remedied by the HOG and CN features. So, our algorithm provides a framework which combines two trackers together and makes them complement each other in visual tracking. And to the best of our knowledge, our algorithm is also the first one which designs an evaluation criterion using correlation filter and zero padding to evaluate the tracking result. Comprehensive experiments are conducted on the Online Tracking Benchmark (OTB), Temple Color (TC128), Benchmark for UAV Tracking (UAV-123), and Visual Object Tracking (VOT) Benchmark. The results show that our algorithm achieves quite a competitive performance when compared with the baseline tracker and several other state-of-the-art trackers.

Visual Object Multimodality Tracking Based on Correlation Filters for Edge Computing

In recent years, visual object tracking has become a very active research field which is mainly divided into the correlation filter-based tracking and deep learning (e.g., deep convolutional neural network and Siamese neural network) based tracking. For target tracking algorithms based on deep learning, a large amount of computation is required, usually deployed on expensive graphics cards. However, for the rich monitoring devices in the Internet of Things, it is difficult to capture all the moving targets in each device in real time, so it is necessary to perform hierarchical processing and use tracking based on correlation filtering in insensitive areas to alleviate the local computing pressure. In sensitive areas, upload the video stream to a cloud computing platform with a faster computing speed to perform an algorithm based on deep features. In this paper, we mainly focus on the correlation filter-based tracking. In the correlation filter-based tracking, the discriminative scale space tracker (DSST) is one of the most popular and typical ones which is successfully applied to many application fields. However, there are still some improvements that need to be further studied for DSST. One is that the algorithms do not consider the target rotation on purpose. The other is that it is a very heavy computational load to extract the histogram of oriented gradient (HOG) features from too many patches centered at the target position in order to ensure the scale estimation accuracy. To address these two problems, we introduce the alterable patch number for target scale tracking and the space searching for target rotation tracking into the standard DSST tracking method and propose a visual object multimodality tracker based on correlation filters (MTCF) to simultaneously cope with translation, scale, and rotation in plane for the tracked target and to obtain the target information of position, scale, and attitude angle at the same time. Finally, in Visual Tracker Benchmark data set, the experiments are performed on the proposed algorithms to show their effectiveness in multimodality tracking.

Hierarchical Convolutional Features Fusion for Visual Tracking

Hierarchical convolutional features have different impact on the tracking performance, as the higher convolutional layers encode the semantic information of targets and earlier convolutional layers are more precise to localize targets. In this paper, we propose a novel scheme for hierarchical convolutional features fusion for visual tracking. In the proposed scheme, hierarchical convolutional features are first concatenated to form the cascading feature at the feature level, and then a convolutional layer is added to reduce the feature dimension. Discriminative correlation filter (DCF) is finally utilized to obtain the target location, which is treated as a differentiable layer in the neural network. The experimental results demonstrate that our proposed scheme achieves superior performances on the visual tracking benchmark.

Boundary‐aware vehicle tracking upon UAV

Discriminative correlation filters (DCFs) have recently achieved competitive performance in visual tracking benchmarks. However, such trackers perform poorly when the target undergoes occlusion, viewpoint variation or other challenging attributes. To tackle these issues, in this Letter, the authors combine the fast DCF trackers with the precise deep learning methods to eliminate the accumulating drift for the vehicle tracking based on unmanned aerial vehicle platform. Specifically, the authors employ the tracking result of the DCF tracker as the input of the boundary regressing network. After judging the existence of the target in the input patch, the proposed network would estimate the boundary of the target vehicle. Furthermore, the output would be updated to the tracking template, aiming at eliminating the accumulation errors and achieving a long-term tracking. The effectiveness of the proposed algorithm is validated through experimental comparison on widely used tracking benchmark data sets.

Occluded Object Tracking System (OOTS)

Visual object tracking remains a challenge facing an intelligent control system. A variety of applications serve many purposes such as surveillance. The developed technology faces plenty of obstacles that should be addressed including occlusion. In visual tracking, online learning techniques are most common due to their efficiency for most video sequences. Many object tracking techniques have emerged. However, the drifting problem in the case of noisy updates has been a stumbling block for the majority of relevant techniques. Such a problem can now be surmounted through updating the classifiers. The proposed system is called the Occluded Object Tracking System (OOTS) It is a hybrid system constructed from two algorithms: a fast technique Circulant Structure Kernels with Color Names (CSK-CN) and an efficient algorithm occlusion-aware Real-time Object Tracking (ROT). The proposed OOTS is evaluated with standard visual tracking benchmark databases. The experimental results proved that the proposed OOTS system is more reliable and provides efficient tracking results than other compared methods.

Learning spatial-temporally regularized complementary kernelized correlation filters for visual tracking

Despite excellent performance shown by spatially regularized discriminative correlation filters (SRDCF) for visual tracking, some issues remain open that hinder further boosting their performance: first, SRDCF utilizes multiple training images to formulate its model, which makes it unable to exploit the circulant structure of the training samples in learning, leading to high computational burden; second, SRDCF is unable to efficiently exploit the powerfully discriminative nonlinear kernels, further negatively affecting its performance. In this paper, we present a novel spatial-temporally regularized complementary kernelized CFs (STRCKCF) based tracking approach. First, by introducing spatial-temporal regularization to the filter learning, the STRCKCF formulates its model with only one training image, which can not only facilitate exploiting the circulant structure in learning, but also reasonably approximate the SRDCF with multiple training images. Furthermore, by incorporating two types of kernels whose matrices are circulant, the STRCKCF is able to fully take advantage of the complementary traits of the color and HOG features to learn a robust target representation efficiently. Besides, our STRCKCF can be efficiently optimized via the alternating direction method of multipliers (ADMM). Extensive evaluations on OTB100 and VOT2016 visual tracking benchmarks demonstrate that the proposed method achieves favorable performance against state-of-the-art trackers with a speed of 40 fps on a single CPU. Compared with SRDCF, STRCKCF provides a 8 × speedup and achieves a gain of 5.5% AUC score on OTB100 and 8.4% EAO score on VOT2016.

Multi-features guided robust visual tracking

This paper focuses on dealing with the tracking challenges such as target occlusion and deformation. It proposes a new tracking method via extracting and evaluating multi-features for both target region and its adjacent surroundings. The multi-features separately describe the key factors to detect target including the color feature, the shape and contour feature, and the distributions of structure and intensity described by the Pearson Correlation Coefficient. These multi-features are proposed as the basic representation of target template and candidates and used to define a matching algorithm between them. The best matched candidate is taken as the final tracking result. To improve the efficiency of target template and candidates, the region of importance (ROI) for target is proposed by evaluating the distribution of salient values on many extended regions. The ROIs produce more accurate regions to form target template and candidates. Finally, a new template update method is defined based on the precision of tracked result to adapt to target state and achieve the follow target tracking. Using 25 videos in visual tracking benchmark, we achieve the quantitative and qualitatively evaluations of 12 different trackers. Many experiments demonstrate that our tracker produces much better results than the present trackers in dealing with target occlusion, deformation, rotation, background clutters.

Visual Object Tracking Based on Mutual Learning Between Cohort Multiscale Feature-Fusion Networks With Weighted Loss

The deep convolutional neural network (CNN) based tracking-by-detection framework recently has become one of the most popular trackers. However, these methods are either time consuming or have greatly reduced performance. This article aims to achieve nearly identical tracking accuracy with the state-of-the-art CNN tracking-by-detection algorithm with relatively faster speed. We study the existing excellent trackers under the CNN tracking-by-detection framework and introduce the following: a multiscale feature pyramid fusion neural network based on dilated convolutions is constructed to learn a scale-invariant discriminative representation for tracking small objects, a hard-threshold weighted cross-entropy loss function is proposed to decrease the gap between object classification and tracking, and a mutual learning-based training policy is used to fuse the information from the network trained by image patches with different contextual regions to further improve the tracking performance. We conduct comprehensive experiments on visual object tracking benchmarks that validate the achievement of competitive performance of the proposed tracker with relatively faster speed both on qualitative and quantitative criteria. Additionally, the experimental results reveal that the proposed mutual learning-based training policy can accelerate the convergence speed and achieve better generalization performance.

Multi-part and scale adaptive visual tracker based on kernel correlation filter

Accurate visual tracking is a challenging issue in computer vision. Correlation filter (CF) based methods are sought in visual tracking based on their efficiency and high performance. Nonetheless, CF-based trackers are sensitive to partial occlusion, which may reduce their overall performance and even lead to failure in tracking challenge. In this paper, we presented a very powerful tracker based on the kernelized correlation filter tracker (KCF). Firstly, we employ an intelligent multi-part tracking algorithm to improve the overall capability of correlation filter based tracker, especially in partial-occlusion challenges. Secondly, to cope with the problem of scale variation, we employ an effective scale adaptive scheme, which divided the target into four patches and computed the scale factor by finding the maximum response position of each patch via kernelized correlation filter. With this method, the scale computation was transformed into locating the centers of the patches. Thirdly, because the small deviation of the central function value will bring the problem of location ambiguity. To solve this problem, the new Gaussian kernel functions are introduced in this paper. Experiments on the default 51 video sequences in Visual Tracker Benchmark demonstrate that our proposed tracker provides significant improvement compared with the state-of-art trackers.

A quantitative attribute-based benchmark methodology for single-target visual tracking

In the past several years, various visual object tracking benchmarks have been proposed, and some of them have been used widely in numerous recently proposed trackers. However, most of the discussions focus on the overall performance, and cannot describe the strengths and weaknesses of the trackers in detail. Meanwhile, several benchmark measures that are often used in tests lack convincing interpretation. In this paper, 12 frame-wise visual attributes that reflect different aspects of the characteristics of image sequences are collated, and a normalized quantitative formulaic definition has been given to each of them for the first time. Based on these definitions, we propose two novel test methodologies, a correlation-based test and a weight-based test, which can provide a more intuitive and easier demonstration of the trackers’ performance for each aspect. Then these methods have been applied to the raw results from one of the most famous tracking challenges, the Video Object Tracking (VOT) Challenge 2017. From the tests, most trackers did not perform well when the size of the target changed rapidly or intensely, and even the advanced deep learning based trackers did not perfectly solve the problem. The scale of the targets was not considered in the calculation of the center location error; however, in a practical test, the center location error is still sensitive to the targets’ changes in size.

LSTM guided ensemble correlation filter tracking with appearance model pool

Deep learning based visual trackers have the potential to provide good performance for object tracking. Most of them use hierarchical features learned from multiple layers of a deep network. However, issues related to deterministic aggregation of these features from various layers, difficulties in estimating variations in scale or rotation of the object being tracked, as well as challenges in effectively modeling the object’s appearance over long time periods leaves substantial scope to improve performance. In this paper, we propose a tracker that learns correlation filters over features from multiple layers of a VGG network. A correlation filter for an individual layer is used to predict the target location. We adaptively learn the contribution of an ensemble of correlation filters for the final location estimation using an LSTM. An adaptive approach is advantageous as different layers encode diverse feature representations and a uniform contribution would not fully exploit this contrastive information. To this end, we use an LSTM as it encodes the interactions for past appearances which is useful for tracking. Further, the scale and rotation parameters are estimated using respective correlation filters. Additionally, an appearance model pool is used that prevents the correlation filter from drifting. Experimental results achieved on five public datasets — Object Tracking Benchmark (OTB100), Visual Object Tracking (VOT) Benchmark 2016, VOT Benchmark 2017, Tracking Dataset and UAV123 Dataset, reveal that our approach outperforms state of the art approaches for object tracking.

Novel visual tracking approach via ant lion optimiser

Ant lion optimiser (ALO) is a new nature-inspired swarm intelligence optimisation algorithm that mimics the hunting mechanism of antlions in nature. ALO has been proved to have the merits of high exploitation and convergence speed benefiting from adaptive boundary shrinking mechanism and elitism. In this work, visual tracking is expressed as searching for object in whole search space by interaction between antlions and ants. A novel ALO-based visual tracking framework is proposed and the adaptation and sensitivity of the parameters in ALO are discussed to improve tracking performance. In addition, considering that ALO tracker needs a lot of iteration consumption, kernel correlation filter with deep feature is integrated into the ALO tracking framework (ALOKCF) to improve track efficiency. Extensive experimental results prove that the ALO tracker is very competitive compared to other trackers, especially for abrupt motion tracking. At the same time, two visual tracking benchmarks are used to verify ALOKCF tracker achieves state-of-the-art performance.

Uncertain motion tracking based on convolutional net with semantics estimation and region proposals

In real world, most of the tracking methods suffer from uncertain motion, which may make the tracker failure because of a local search window with the motion smooth assumption. To address this problem, a novel tracking framework based on convolutional net with semantics estimation and region proposals is proposed. Firstly, we present a semantics object proposals generation strategy, including category-level semantics proposals, one-object-level semantics estimation and semantics-contextual proposals generation, to obtain a few of high-quality object-oriented proposals covering uncertain motion. Secondly, combining the globally sparse semantics region proposals prediction and correlation filter prediction, a hybrid semantics tracking algorithm is proposed, which obtains a coarse object location by the decision of multiple response maps. Finally, we learn and train independent correlation filter to estimate the scale of target for a higher tracking accuracy. Extensive experiments on two visual tracking benchmarks and results demonstrate our method achieves state-of-the-art performance.

Benchmarking Deep Trackers on Aerial Videos.

In recent years, deep learning-based visual object trackers have achieved state-of-the-art performance on several visual object tracking benchmarks. However, most tracking benchmarks are focused on ground level videos, whereas aerial tracking presents a new set of challenges. In this paper, we compare ten trackers based on deep learning techniques on four aerial datasets. We choose top performing trackers utilizing different approaches, specifically tracking by detection, discriminative correlation filters, Siamese networks and reinforcement learning. In our experiments, we use a subset of OTB2015 dataset with aerial style videos; the UAV123 dataset without synthetic sequences; the UAV20L dataset, which contains 20 long sequences; and DTB70 dataset as our benchmark datasets. We compare the advantages and disadvantages of different trackers in different tracking situations encountered in aerial data. Our findings indicate that the trackers perform significantly worse in aerial datasets compared to standard ground level videos. We attribute this effect to smaller target size, camera motion, significant camera rotation with respect to the target, out of view movement, and clutter in the form of occlusions or similar looking distractors near tracked object.