Visual Tracking Benchmark Research Articles

Classifier Adaptive Fusion: Deep Learning for Robust Outdoor Vehicle Visual Tracking

Deep auto-encoder (DAE) models have been successfully used in object tracking due to its strong capability of feature representation. However, single deep auto-encoder model would not be robust enough to represent the appearance model of outdoor vehicle for its harsh working environment, such as illumination variation, occlusion, cluttered background and so on. In this paper, a novel multiple-DAE-based tracking approach, that is, classifier adaptive fusion for robust outdoor vehicle visual tracking approach is proposed under particle filter framework. Firstly, two deep auto-encoders are offline trained by gray-scale image and gradient image of the raw training images, respectively to obtain the stronger feature representation of gray-scale image and gradient image. Secondly, two classifiers are constructed using the encoder of the two well-trained deep auto-encoders and the output of the each classifier is used to compute the confidence of the corresponding particles. Finally, the confidence output of the two classifiers is fused and applied in online tracking, where, the fusion weight of the each classifier is computed according to the distribution of particles represented by different classifier. Extensive tracking experiments conducted on visual tracking benchmark (VTB) show that the proposed tracking algorithm outperforms 9 popular tracking algorithms in the challenge scenes of outdoor vehicle tracking such as illumination variation, occlusion, cluttered background and scale variation.

Cascaded Iterative Training Model and Parallel Multi-Classifiers for Visual Object Tracking

Because of the superiority in distinguishing the target from the background by learning mechanism, discriminative classifier attracts a great deal of attention in target detection and tracking systems. Generally, a discriminative classifier is learned from the initial image patch. However, the other algorithms, using only one image frame inhibit the performance of the classifier. A novel training strategy of cascade iteration, which is based on the multiple image frames, is used to initialize the discriminative classifier for the purpose of weakening the initialization limitation and improving the algorithms' adaptability and robustness. Furthermore, to cope with the changes in natural images, online learning is usually adopted to enhance the performance of classifiers, but it easily leads to tracking failures, especially in the case of full occlusion. In order to attenuate the influence of full occlusion, a multi-classifiers parallel tracking algorithm is proposed by designing the multi-classifiers and inducing extra constraints. The experiments are performed on the visual tracking benchmark, and the proposed algorithm is more robust while the target is fully occluded. The simulation results show that the proposed algorithm exhibits better performance compared to the state-of-the-art trackers, especially with KCF, TLD, and STRUCK.

Adaptive Sample Weight for Machine Learning Computer Vision Algorithms in V2X Systems

In machine learning, training sample set management has an important impact on the performance of visual detection and tracking algorithms, as corrupted training samples degrade the tracking performance, especially in practical scenarios such as vehicular networks. However, how to evaluate and remove the corrupted training samples still remains a challenging topic. In this paper, we propose a novel scheme to remove the corrupted training samples in visual tracking, which will improve the tracking performance dramatically. In the proposed scheme, a novel training sample set management method based on the adaptive sample weight is presented. Specifically, similarity learning is first utilized to evaluate the quality of training samples with similarity score. Then, if the similarity score is below a certain threshold, the training sample is deemed as the corrupted one and is removed from the training sample set. The experimental results show that the proposed scheme obtains superior performances on visual tracking benchmarks and vehicular scenarios.

One-Shot Scale and Angle Estimation for Fast Visual Object Tracking

Visual object tracking is the process of tracking an arbitrary object in a video, where the bounding box of the object is given in the first frame. Siamese network-based visual object tracking approaches have recently received considerable attention for their high speed and superior performance. However, for scale and angle estimation, Siamese trackers require multiple search regions, which increases the computation time, thereby decreasing the real-time tracking performance. This paper proposes a one-shot Siamese network, named Siam-OS, for fast and efficient visual object tracking. Siam-OS uses only a single search region and estimates the scale and angle of the target bounding box. This significantly reduces the number of computations required for the deep convolutional feature extraction, and thus increases the tracking speed. The experimental results with Visual Object Tracking (VOT) benchmarks show the effectiveness of the proposed Siam-OS in terms of the accuracy, robustness, expected average overlap, and speed.

Learning an Orientation and Scale Adaptive Tracker With Regularized Correlation Filters

Spatial and temporal constraints are very important for correlation filter (CF)-based trackers. However, the existing methods usually fail to regularize the CF learning by the spatio-temporal information because of the ineffective target representation. To address the issue, we propose a novel Orientation and Scale adaptive tracker with Regularized Correlation Filters (OSRCF) for visual tracking. First, we directly employ the target region to build a spatio-temporally regularized CF, which is solved efficiently by the alternating direction method of multipliers (ADMM). Especially, the spatial regularization module can alleviate boundary effects by suppressing the outside background, while the temporal one can handle rapid appearance changes by smoothing the CF updating. Second, we obtain a non-axis-aligned bounding box for the target region representation by orientation and scale adaptive strategy, which cooperates a straightforward orientation estimation with a discriminative scale space correlation filter. We perform comprehensive experiments on two recent visual tracking benchmark datasets: VOT2017 and OTB2015. The results show that our OSRCF tracker outperforms top-ranked methods with handcrafted features in VOT2017 and achieves outstanding performance compared to some state-of-the-art methods in OTB2015.

Robust visual tracking with adaptive initial configuration and likelihood landscape analysis

Here, the authors propose a novel tracking algorithm that can automatically modify the initial configuration of a target to improve the tracking accuracy in subsequent frames. To achieve this goal, the authors’ method analyses the likelihood landscape (LL) for the image patch described by the initial configuration. A good configuration has a unimodal distribution with a steep shape in the LL. Using the LL analysis, the authors’ method improves the initial configuration, resulting in more accurate tracking results. The authors improve the conventional LL analysis based on two ideas. First, the authors’ method analyses the LL in the RGB space rather than the grey space. Second, the method introduces an additional criterion for a good configuration: a high likelihood value at the mode. The authors further enhance their method through post‐processing of the visual tracking results at each frame, where the estimated bounding boxes are modified by the LL analysis. The experimental results demonstrate that the authors’ advanced LL analysis helps improve the tracking accuracy of several baseline trackers on a visual tracking benchmark data set. In addition, the authors’ simple post‐processing technique significantly enhances the visual tracking performance in terms of precision and success rate.

Real-Time Inland CCTV Ship Tracking

The predator algorithm is a representative pioneering work that achieves state-of-the-art performance on several popular visual tracking benchmarks and with great success when commercially applied to real-time face tracking in long-term unconstrained videos. However, there are two major drawbacks of predator algorithm when applied to inland CCTV (closed-circuit television) ship tracking. First, the LK short-term tracker within predator algorithm easily tends to drift if the target ship suffers partial or even full occlusion, mainly because the corner-points-like features employed by LK tracker are very sensitive to occlusion appearance change. Second, the cascaded detector within the predator algorithm searches for candidate objects in a predefined scale set, usually including 3-5 elements, which hampers the tracker to adapt to the potential diverse scale variations of the target ship. In this paper, we design a random projection based short-term tracker which can dramatically ease the tracking drift when the ship is under occlusion. Furthermore, a forward-backward feedback mechanism is proposed to estimate the scale variation between two consecutive frames. We prove that these two strategies gain significant improvements over the predator algorithm and also show that the proposed method outperforms several other state-of-the-art trackers.

Appearance features in Encoding Color Space for visual surveillance

Abstract Person re-identification and visual tracking are two important tasks in video surveillance. Many works have been done on appearance modeling for these two tasks. However, existing feature descriptors are mainly constructed on three-channel color spaces, such like RGB, HSV and XYZ. These color spaces somehow enable meaningful representation for color, yet may lack distinctiveness for real-world tasks. In this paper, we propose a multi-channel Encoding Color Space (ECS), and consider the color distinction with the design of image feature descriptor. In order to overcome the illumination variation and shape deformation, we design features on the basis of the Encoding Color Space and Histogram of Oriented Gradient (HOG), which enables rich color-gradient characteristics. Additionally, we extract Second Order Histogram (SOH) on the descriptor constructed to capture abstract information with layout constrains. Exhaustive experiments are performed on datasets VIPeR, CAVIAR, CUHK01 and Visual Tracking Benchmark. Experimental results on these datasets show that our feature descriptors could achieve promising performance.

Learning spatio-temporal context via hierarchical features for visual tracking

Spatio-temporal context (STC) based visual tracking algorithms have demonstrated remarkable tracking capabilities in recent years. In this paper, we propose an improved STC method, which seamlessly integrates capabilities of the powerful feature representations and mappings from the convolutional neural networks (CNNs) based on the theory of transfer learning. Firstly, the dynamic training confidence map, obtained from a mapping neural network using transferred CNN features, rather than the fixed training confidence map is utilized in our tracker to adapt the practical tracking scenes better. Secondly, we exploit hierarchical features from both the original image intensity and the transferred CNN features to construct context prior models. In order to enhance the accuracy and robustness of our tracker, we simultaneously transfer the fine-grained and semantic features from deep networks. Thirdly, we adopt the training confidence index (TCI), reflected from the dynamic training confidence map, to guide the updating process. It can determine whether back propagations should be conducted in the mapping neural network, and whether the STC model should be updated. The introduction of the dynamic training confidence map could effectively deal with the problem of location ambiguity further in our tracker. Overall, the comprehensive experimental results illustrate that the tracking capability of our tracker is competitive against several state-of-the-art trackers, especially the baseline STC tracker, on the existing OTB-2015 and UAV123 visual tracking benchmarks.

Action detection by double hierarchical multi-structure space–time statistical matching model

Aimed at the complex information in videos and low detection efficiency, an actions detection model based on neighboring Gaussian structure and 3D LARK features is put forward. We exploit a double hierarchical multi-structure space–time statistical matching model (DMSM) in temporal action localization. First, a neighboring Gaussian structure is presented to describe the multi-scale structural relationship. Then, a space–time statistical matching method is proposed to achieve two similarity matrices on both large and small scales, which combines double hierarchical structural constraints in model by both the neighboring Gaussian structure and the 3D LARK local structure. Finally, the double hierarchical similarity is fused and analyzed to detect actions. Besides, the multi-scale composite template extends the model application into multi-view. Experimental results of DMSM on the complex visual tracker benchmark data sets and THUMOS 2014 data sets show the promising performance. Compared with other state-of-the-art algorithm, DMSM achieves superior performances.

Dual linear structured support vector machine tracking method via scale correlation filter

Adaptive tracking-by-detection methods based on structured support vector machine (SVM) performed well on recent visual tracking benchmarks. However, these methods did not adopt an effective strategy of object scale estimation, which limits the overall tracking performance. We present a tracking method based on a dual linear structured support vector machine (DLSSVM) with a discriminative scale correlation filter. The collaborative tracker comprised of a DLSSVM model and a scale correlation filter obtains good results in tracking target position and scale estimation. The fast Fourier transform is applied for detection. Extensive experiments show that our tracking approach outperforms many popular top-ranking trackers. On a benchmark including 100 challenging video sequences, the average precision of the proposed method is 82.8%.

Real-time fast moving object tracking in severely degraded videos captured by unmanned aerial vehicle

Object tracking for unmanned aerial vehicle applications in outdoor scenes is a very complex problem. In videos captured by unmanned aerial vehicle, due to frequent variation in illumination, motion blur, image noise, deformation, lack of image texture, occlusion, fast motion, and other degradations, most tracking methods will lead to failure. The article focuses on the object tracking in severely degraded videos. To deal with those various degradations, a real-time object tracking method for high dynamic background is developed. By integrating histogram of oriented gradient, RGB histogram and motion histogram into a novel statistical model, our method can robustly track the target in unmanned aerial vehicle captured videos. Compared to those existing methods, our proposed approach costs less resource in the tracking, significantly increases the tracking speed, and runs faster than state-of-the-art methods. Also, our approach achieved satisfactory tracking results on the challenging visual tracking benchmark, object tracking benchmark 2013, the supplementary experiments demonstrates that our method is more effective and accurate than other methods.

Complementary Tracking via Dual Color Clustering and Spatio-Temporal Regularized Correlation Learning

Recently, a simple, yet effective and efficient tracker named Staple has achieved promising performance in terms of efficiency and accuracy on a series of visual tracking benchmarks. Staple is equipped with complementary learners of discriminative correlation filters (DCFs) and color histograms, which are robust to both color changes and deformations. However, it has some drawbacks: 1) Staple only employs standard color histograms with the same quantization step for all sequences, which does not consider the specific structural information of target in each sequence, thereby affecting its discriminative capability to separate target from background. 2) The standard DCFs are efficient but suffer from unwanted boundary effects, leading to failures in some challenging scenarios. To address these issues, we present a dual color clustering and spatio-temporal regularized correlation regressions-based complementary tracker (CSCT). The proposed CSCT includes two components with complementary merits to adaptively deal with significant color variations and deformations for each sequence: First, we design a novel color clustering-based histogram model that first adaptively divides the colors of the target in the 1st frame into several cluster centers, and then the cluster centers are taken as references to construct adaptive color histograms for targets in the coming frames, which enable to adapt significant target deformations. Second, we propose to learn spatio-temporal regularized CFs, which not only enable to avoid boundary effects but also provides a more robust appearance model than the discriminative CFs in Staple in the case of large appearance variations. Compared to Staple, our CSCT with handcrafted features achieves a gain of 5.9%, 3.4%, and 1.5% on OTB100, Temple-Color, and VOT2016 benchmarks in terms of AUC and EAO scores, respectively. Moreover, our CSCT performs favorably against several state-of-the-art trackers, including the deep learning-based trackers.

An Improved Kernelized Correlation Filter with the Histogram in Hue-Saturation-Value Color Space for Object Tracking

Visual tracking is a significant problem in computer vision. It requires robustness and real-time. Correlation Filter has achieved state-of-the-art results on this problem. It utilizes the circulant structure of matrix and runs at a high frame rate. However, the circulant structure of matrix is severely impacted by the background, deformation and illumination variation. Therefore, it will generate drift when the algorithm determines the final position. In this paper, we propose a strategy to tackle drift problem. We introduce the histogram of Hue-Saturation-Value (HSV) color space into the kernelized correlation filter in order to obtain a new position for redetection. Finally, we evaluate our algorithm on online tracking benchmark(OTB) and a visual object tracking benchmark(VOT). On the two datasets, the proposed approach achieves an improvement compared with the original tracker and achieves promising performance compared with other state-of-the-art trackers.

Tracking using Numerous Anchor Points

In this paper, an online adaptive model-free tracker is proposed to track single objects in video sequences to deal with real-world tracking challenges like low-resolution, object deformation, occlusion and motion blur. The novelty lies in the construction of a strong appearance model that captures features from the initialized bounding box and then are assembled into anchor point features. These features memorize the global pattern of the object and have an internal star graph-like structure. These features are unique and flexible and help tracking generic and deformable objects with no limitation on specific objects. In addition, the relevance of each feature is evaluated online using short-term consistency and long-term consistency. These parameters are adapted to retain consistent features that vote for the object location and that deal with outliers for long-term tracking scenarios. Additionally, voting in a Gaussian manner helps in tackling inherent noise of the tracking system and in accurate object localization. Furthermore, the proposed tracker uses pairwise distance measure to cope with scale variations and combines pixel-level binary features and global weighted color features for model update. Finally, experimental results on a visual tracking benchmark dataset are presented to demonstrate the effectiveness and competitiveness of the proposed tracker.

Graph Regularized and Locality-Constrained Coding for Robust Visual Tracking

Visual tracking is complicated due to factors, such as occlusion, background clutter, abrupt target motion, and illumination variations, among others. In recent years, subspace representation and sparse coding techniques have demonstrated significant improvements in tracking. However, performance gain in tracking has been at the expense of losing locality and similarity attributes among the instances to be encoded. In this paper, a graph regularized and locality-constrained coding (GRLC) technique that encapsulates local manifold structure of the data in order to preserve locality and similarity information among instances is proposed. The GRLC methodology incorporates a similarity-preserving term within the objective function of the locality-constrained linear coding model, thereby overcoming some of the inherent instability issues common to such coding methods. In the proposed GRLC scheme, a graph Laplacian regularizer is chosen as a smoothing operator to learn both the representation dictionary and the coefficients by preserving the local structure of the data. This graph Laplacian smoothing operator ensures that the representations vary smoothly along the geodesics of the data manifold. Thus, by deriving the objective function of the GRLC method, a discriminative dictionary of instances can be iteratively obtained and the corresponding coefficients for each candidate can be computed using this learned dictionary. Finally, an effective observation likelihood function based on reconstruction error and a simple dictionary update scheme for visual target tracking are also proposed. Experimental results on the CVPR2013 visual tracker benchmark have demonstrated a favorable performance of the proposed technique both in terms of accuracy and robustness.

Visual tracking with multiple Hough detectors

We propose a visual tracking method using multiple Hough detectors to address the problem of long-term robust object tracking in unconstrained environments. The method constructs the detectors based on the feature selection by the mutual information. These detectors serve to learn the partial appearances of target and synchronously evaluate image locations via the voting based detection with the generalized Hough transform. According to the result of detections, the best detector is selected by the minimum entropy criterion and delivers the final hypotheses for target location. The feature selection allows our tracker to be able to obtain and use the most discriminative parts of target and thus more robust to its changes, e.g. occlusion and deformation. The detector selection can correct undesirable model updates and restore the tracker after tracking failure. Meanwhile, the Hough-based detection can reduce the amount of noise introduced during online self-training and thus effectively prevent the tracker from drifting. The method is evaluated on the CVPR2013 Visual Tracker Benchmark and the experimental results demonstrate our method outperforms other tracking algorithms in terms of both success rate and precision.

Simple Online Multiple Human Tracking based on LK Feature Tracker and Detection for Embedded Surveillance

In this paper, we propose a simple online multiple object (human) tracking method, LKDeep (Lucas-Kanade feature and Detection based Simple Online Multiple Object Tracker), which can run in fast online enough on CPU core only with acceptable tracking performance for embedded surveillance purpose. The proposed LKDeep is a pragmatic hybrid approach which tracks multiple objects (humans) mainly based on LK features but is compensated by detection on periodic times or on necessity times. Compared to other state-of-the-art multiple object tracking methods based on ‘Tracking-By-Detection (TBD)’ approach, the proposed LKDeep is faster since it does not have to detect object on every frame and it utilizes simple association rule, but it shows a good object tracking performance. Through experiments in comparison with other multiple object tracking (MOT) methods using the public DPM detector among online state-of-the-art MOT methods reported in MOT challenge [1], it is shown that the proposed simple online MOT method, LKDeep runs faster but with good tracking performance for surveillance purpose. It is further observed through single object tracking (SOT) visual tracker benchmark experiment [2] that LKDeep with an optimized deep learning detector can run in online fast with comparable tracking performance to other state-of-the-art SOT methods.

Deterministic object tracking using Gaussian ringlet and directional edge features

Challenges currently existing for intensity-based histogram feature tracking methods in wide area motion imagery (WAMI) data include object structural information distortions, background variations, and object scale change. These issues are caused by different pavement or ground types and from changing the sensor or altitude. All of these challenges need to be overcome in order to have a robust object tracker, while attaining a computation time appropriate for real-time processing. To achieve this, we present a novel method, Directional Ringlet Intensity Feature Transform (DRIFT), which employs Kirsch kernel filtering for edge features and a ringlet feature mapping for rotational invariance. The method also includes an automatic scale change component to obtain accurate object boundaries and improvements for lowering computation times. We evaluated the DRIFT algorithm on two challenging WAMI datasets, namely Columbus Large Image Format (CLIF) and Large Area Image Recorder (LAIR), to evaluate its robustness and efficiency. Additional evaluations on general tracking video sequences are performed using the Visual Tracker Benchmark and Visual Object Tracking 2014 databases to demonstrate the algorithms ability with additional challenges in long complex sequences including scale change. Experimental results show that the proposed approach yields competitive results compared to state-of-the-art object tracking methods on the testing datasets.

Robust Visual Tracking With Multitask Joint Dictionary Learning

Dictionary learning for sparse representation has been increasingly applied to object tracking, however, the existing methods only utilize one modality of the object to learn a single dictionary. In this paper, we propose a robust tracking method based on multitask joint dictionary learning. Through extracting different features of the target, multiple linear sparse representations are obtained. Each sparse representation can be learned by a corresponding dictionary. Instead of separately learning the multiple dictionaries, we adopt a multitask learning approach to learn the multiple linear sparse representations, which provide additional useful information to the classification problem. Because different tasks may favor different sparse representation coefficients, yet the joint sparsity may enforce the robustness in coefficient estimation. During tracking, a classifier is constructed based on a joint linear representation, and the candidate with the smallest joint decision error is selected to be the tracked object. In addition, reliable tracking results and augmented training samples are accumulated into two sets to update the dictionaries for classification, which helps our tracker adapt to the fast time-varying object appearance. Both qualitative and quantitative evaluations on CVPR2013 visual tracking benchmark demonstrate that our method performs favorably against state-of-the-art trackers.