Correlation Filter-based Tracker Research Articles

Adaptive and Anti-Drift Motion Constraints for Object Tracking in Satellite Videos

Object tracking in satellite videos has garnered significant attention due to its increasing importance. However, several challenging attributes, such as the presence of tiny objects, occlusions, similar objects, and background clutter interference, make it a difficult task. Many recent tracking algorithms have been developed to tackle these challenges in tracking a single interested object, but they still have some limitations in addressing them effectively. This paper introduces a novel correlation filter-based tracker, which uniquely integrates attention-enhanced bounding box regression and motion constraints for improved single-object tracking performance. Initially, we address the regression-related interference issue by implementing a spatial and channel dual-attention mechanism within the search area’s region of interest. This enhancement not only boosts the network’s perception of the target but also improves corner localization. Furthermore, recognizing the limitations in small size and low resolution of target appearance features in satellite videos, we integrate motion features into our model. A long short-term memory (LSTM) network is utilized to create a motion model that can adaptively learn and predict the target’s future trajectory based on its historical movement patterns. To further refine tracking accuracy, especially in complex environments, an anti-drift module incorporating motion constraints is introduced. This module significantly boosts the tracker’s robustness. Experimental evaluations on the SatSOT and SatVideoDT datasets demonstrate that our proposed tracker exhibits significant advantages in satellite video scenes compared to other recent trackers for common scenes or satellite scenes.

Motion-aware object tracking for aerial images with deep features and discriminative correlation filter

AbstractObject tracking is a challenging task which is required for different problems such as surveillance, traffic analysis and human-computer interaction. The problem of tracking an object can be considered in different categories such as single object tracking, multiple object tracking, short-term tracking, long-term tracking, tracking by detection and detection-free tracking. This study focuses on detection-free tracking for ground targets on aerial images. The studies in the literature show that correlation filter and deep learning based object trackers perform well recently. This paper proposes a new correlation filter-based tracker containing a strategy for re-detection issue. We improve the performance of correlation filter-based tracker by adding a lightweight re-detection ability to the correlation filter tracker in case of a long occlusion or complete loss of target. We use deep features to train Discriminative Correlation Filter(DCF) by integrating sub-networks from pre-trained ResNet and SAND models. The experimental results on the popular UAV123L dataset show that the proposed method(MADCF) improves the performance of DCF tracker and have a reasonable performance for long-term tracking problem. Moreover, we prepare a new tracking dataset (PESMOD tracking) consisting of UAV images, and we evaluate the proposed method and state-of-the-art method in this dataset. We observed that the proposed method performs much better in ground target tracking from VIVID and PESMOD aerial datasets. The proposed MADCF tracker performs better for small targets tracked by UAVs compared to the deep learning-based trackers. The source code and prepared dataset are available at http://github.com/mribrahim/MADCF

All-Day Object Tracking for Unmanned Aerial Vehicle

Unmanned aerial vehicle (UAV) has facilitated a wide range of real-world applications and attracted extensive research in the mobile computing field. Specially, developing real-time robust visual onboard trackers for all-day aerial maneuver can remarkably broaden the scope of intelligent deployment of UAV. However, prior tracking methods have merely focused on robust tracking in the well-illuminated scenes, while ignoring trackers capabilities to be deployed in the dark. In darkness, the conditions can be more complex and harsh, easily posing inferior robust tracking or even tracking failure. To this end, this work proposes a novel discriminative correlation filter-based tracker with illumination adaptive and anti-dark capability, namely ADTrack. ADTrack firstly exploits image illuminance information to enable adaptability of the model to the given light condition. Then, by virtue of an efficient enhancer, ADTrack carries out image pretreatment where a target aware mask is generated. Benefiting from the mask, ADTrack aims to solve a novel dual regression problem where dual filters are online trained with mutual constraint. Besides, this work also constructs a UAV nighttime tracking benchmark UAVDark135. Exhaustive experiments on authoritative benchmarks and onboard tests are implemented to validate the superiority and robustness of ADTrack in all-day conditions.

High-speed tracking with multi-templates correlation filters

Correlation filter (CF)-based trackers have demonstrated impressive performance on visual tracking benchmark datasets with high frame rates. A lot of research focuses on extracting strong features to learn a robust model of the object’s appearance. However, the circularly shifted samples used in traditional CF training come from one single image patch and have limited information about the target. Inspired by various data augmentation methods in deep learning, we propose a general multi-templates correlation filter-based tracker that expands the CF tracker with multihandcrafted training samples. We reformulate the original optimization problem and provide a closed-form solution to maintain the high-speed calculation. Several methods to generate the additional samples are evaluated. Comprehensive experiments illustrate that our framework obviously improves the tracking performance and is general enough to be incorporated with classical CF trackers.

Multi-experts Joint Decision with Adaptive Model Updater for Robust Visual Tracking

Over these years, correlation filters based trackers have shown edges both in accuracy and speed. However, variations of target appearance caused by heavy occlusion, rotation, background clutters and target deformations are the major challenges for tracking. To solve these problems, many works put on exploiting the power of target representation, such as high-level convolutional features. Nonetheless, these methods make a great compromise between the speed and performance. At the same time, there are few researches on improving the performance of model updater and the ensemble methods. In this paper, a multi-experts joint decision strategy base on kernelized correlation filters is proposed to obtain robust and accurate visual tracking, two trackers with handcrafted features and deep convolutional neural network features are integrated in this framework. We also investigate the mechanism of tracking failure caused by occlusion and background clutters, and propose a novel criterion to evaluate the reliability of samples. Our work includes extending the kernelized correlation filter-based tracker with the capability of handling scale changes as well. The proposed tracker is extensively evaluated on the OTB-2013, OTB-2015 and VOT2015 benchmark datasets. Compared with the state-of-the-art trackers, the distinguished experimental results demonstrate the effectiveness of the proposed framework.

A Robust Correlation Filtering Tracker with Resampling-Detection and Adaptive Fusion Multi-features

Recently, correlation filter is widely used in visual tracking for its robust and accuracy. However, it is still a challenge in tracking with complex situations such as target blurring, occlusion, and scale variation. In this paper, a correlation filter-based tracker with resampling-detection and scale estimation is proposed. We use multiple features with adaptive fusion to describe the target appearance, and resampling-detection module will be performed on the frame which tracking confidence determined by PSR is lower than a threshold. Besides, scale pyramid is introduced to estimate the scale. The extensive experimental evaluates on the OTB benchmark and results show that our approach outperforms the baseline trackers and has excellent performance in accuracy and robust, especially on the challenge of fast motion and motion blur. Additionally, our approach is computationally efficient and suitable for real-time applications.

Long‐term tracking with fast scale estimation and efficient re‐detection

In long-term tracking applications, occlusion and scale variation are common attributes which cause performance degradation. Existing solutions use heavy calculation to deal with these problems, without considering the real-time implementation. Therefore, the authors propose a novel long-term tracker with fast scale estimation and efficient re-detection scheme to maintain real-time speed and favourable accuracy. Specifically, the authors integrate a distance metric method into correlation filter-based tracker to realise fast translation calculation and scale estimation. In addition, the authors advocate a keypoint-matching based confidence indicator to verify the tracking result and activate the re-detection module when the occlusion happens. The authors test our approach on challenging sequences with scale variation and occlusion. Experiments demonstrate that our proposed tracker procures preferable effect than state-of-the-art methods in the aspect of both speed and accuracy.

A Comparison of Correlation Filter-Based Trackers and Struck Trackers

In recent years, two types of trackers, namely correlation filter-based tracker (CF tracker) and structured output tracker (struck), have exhibited the state-of-the-art performance. However, there seems to be a lack of analytic work on their relations in the computer vision community. In this paper, we investigate two state-of-the-art CF trackers, i.e., spatial regularization discriminative correlation filter (SRDCF) and correlation filter with limited boundaries (CFLB), and struck, and reveal their relations. Specifically, after extending the CFLB to its multiple channel versions, we prove the relation between SRDCF and CFLB on the condition that the spatial regularization factor of SRDCF is replaced by the masking matrix of CFLB. We also prove the asymptotical approximate relation between SRDCF and struck on the conditions that the spatial regularization factor of SRDCF is replaced by an indicator function of object bounding box, the weights of SRDCF in its loss item are replaced by those of struck, the linear kernel is employed by struck, and the search region tends to infinity. The extensive experiments on public benchmarks OTB50 and OTB100 are conducted to verify our theoretical results. Moreover, we explain how detailed differences among SRDCF, CFLB, and Struck would give rise to slightly different performances on visual sequences.

Robust Estimation of Similarity Transformation for Visual Object Tracking

Most of existing correlation filter-based tracking approaches only estimate simple axis-aligned bounding boxes, and very few of them is capable of recovering the underlying similarity transformation. To tackle this challenging problem, in this paper, we propose a new correlation filter-based tracker with a novel robust estimation of similarity transformation on the large displacements. In order to efficiently search in such a large 4-DoF space in real-time, we formulate the problem into two 2-DoF sub-problems and apply an efficient Block Coordinates Descent solver to optimize the estimation result. Specifically, we employ an efficient phase correlation scheme to deal with both scale and rotation changes simultaneously in log-polar coordinates. Moreover, a variant of correlation filter is used to predict the translational motion individually. Our experimental results demonstrate that the proposed tracker achieves very promising prediction performance compared with the state-of-the-art visual object tracking methods while still retaining the advantages of high efficiency and simplicity in conventional correlation filter-based tracking methods.

Robust long‐term correlation tracking with multiple models

To address the challenge of repetitive target appearance variation and frequent occlusion, existing visual tracking methods either handle corrupted samples or correct the appearance model. In this study, the authors propose a novel framework that successfully combines these two strategies. In their method, the base tracker is an improved discriminative correlation filter-based tracker, in which an independent classifier is employed to alleviate the problem of corrupted samples; the best model is selected for improvement from a group of models, which they call a ‘model colony’. The model colony is composed of models updated via different processes. The correlation output and the peak-to-sidelobe ratio are used to evaluate each model in the model colony. In addition, they propose a novel criterion called the maximum-to-others ratio for superior model selection. Experiments on 80 challenging sequences show that their tracker outperforms state-of-the-art trackers. In addition, experimental results demonstrate that their formulation significantly improves the performance of their base tracker.

Automatic Individual Pig Detection and Tracking in Pig Farms.

Individual pig detection and tracking is an important requirement in many video-based pig monitoring applications. However, it still remains a challenging task in complex scenes, due to problems of light fluctuation, similar appearances of pigs, shape deformations, and occlusions. In order to tackle these problems, we propose a robust on-line multiple pig detection and tracking method which does not require manual marking or physical identification of the pigs and works under both daylight and infrared (nighttime) light conditions. Our method couples a CNN-based detector and a correlation filter-based tracker via a novel hierarchical data association algorithm. In our method, the detector gains the best accuracy/speed trade-off by using the features derived from multiple layers at different scales in a one-stage prediction network. We define a tag-box for each pig as the tracking target, from which features with a more local scope are extracted for learning, and the multiple object tracking is conducted in a key-points tracking manner using learned correlation filters. Under challenging conditions, the tracking failures are modelled based on the relations between responses of the detector and tracker, and the data association algorithm allows the detection hypotheses to be refined; meanwhile the drifted tracks can be corrected by probing the tracking failures followed by the re-initialization of tracking. As a result, the optimal tracklets can sequentially grow with on-line refined detections, and tracking fragments are correctly integrated into respective tracks while keeping the original identifications. Experiments with a dataset captured from a commercial farm show that our method can robustly detect and track multiple pigs under challenging conditions. The promising performance of the proposed method also demonstrates the feasibility of long-term individual pig tracking in a complex environment and thus promises commercial potential.

Learning to Update for Object Tracking with Recurrent Meta-learner.

Model update lies at the heart of object tracking. Generally, model update is formulated as an online learning problem where a target model is learned over the online training set. Our key innovation is to formulate the model update problem in the meta-learning framework and learn the online learning algorithm itself using large numbers of offline videos, i.e., learning to update. The learned updater takes as input the online training set and outputs an updated target model. As a first attempt, we design the learned updater based on recurrent neural networks (RNNs) and demonstrate its application in a template-based tracker and a correlation filter-based tracker. Our learned updater consistently improves the base trackers and runs faster than realtime on GPU while requiring small memory footprint during testing. Experiments on standard benchmarks demonstrate that our learned updater outperforms commonly used update baselines including the efficient exponential moving average (EMA)-based update and the well-designed stochastic gradient descent (SGD)-based update. Equipped with our learned updater, the template-based tracker achieves state-of-the-art performance among realtime trackers on GPU.

Correlation filter-based visual tracking via holistic and reliable local parts

Visual tracking is a challenging task in computer vision due to various appearance changes of the target. Although correlation filter-based trackers have achieved competitive results, they may easily lead to tracking failure because of the high sensitivity of correlation filter to occlusion. Part-based correlation filter trackers can deal with partial occlusion to some extent, but they may easily drift to the background in the case of fast motion or heavy occlusion. To better solve the above-mentioned problems, a kernelized correlation filter-based tracker that processes both holistic and reliable local parts is proposed. For local parts, reliable parts are identified by peak-to-sidelobe ratio. When all parts are unreliable, we propose to apply a sliding window on each part to generate patches, among which a reliable patch is identified, and the part is replaced by the reliable patch. In holistic level, holistic tracking is performed with the rough position voted by reliable local parts, and then the holistic tracking result is used to provide feedback for parts to update its scale and filter. Moreover, we propose to reset unreliable parts when the holistic tracking result is reliable. The experimental results illustrate that the proposed tracker outperforms those of several state-of-the-art trackers.

A novel framework for robust long-term object tracking in real-time

In this paper, we study the problem of long-term object tracking, where the object may become fully occluded or leave/re-enter the camera view. In this setting, the drifting due to significant appearance change of the object and the recovery from tracking failure are two major issues. To address these two issues, we propose an intelligent framework to integrate a tracker and detector, wherein the tracker module is used to validate the output of the detector with online learning. The key insight of our work is the importance of how a tracker and detector are integrated, which has received little attention in the literature. Based on our proposed framework, a correlation filter-based tracker and a cascaded detector are utilized to implement a robust long-term tracking algorithm. Extensive experimental results show that the proposed framework performs better compared to specific choices of tracker/detector modules and to state-of-the-art tracking-and-detection methods. Additionally, we extend the proposed system with a centralized strategy to achieve cooperative tracking using multiple cameras in a laboratory setting.

Robust Visual Tracking Using Multi-Frame Multi-Feature Joint Modeling

It remains a huge challenge to design effective and efficient trackers under complex scenarios, including occlusions, illumination changes and pose variations. To cope with this problem, a promising solution is to integrate the temporal consistency across consecutive frames and multiple feature cues in a unified model. Motivated by this idea, we propose a novel correlation filter-based tracker in this work, in which the temporal relatedness is reconciled under a multi-task learning framework and the multiple feature cues are modeled using a multi-view learning approach. We demonstrate the resulting regression model can be efficiently learned by exploiting the structure of blockwise diagonal matrix. A fast blockwise diagonal matrix inversion algorithm is developed thereafter for efficient online tracking. Meanwhile, we incorporate an adaptive scale estimation mechanism to strengthen the stability of scale variation tracking. We implement our tracker using two types of features and test it on two benchmark datasets. Experimental results demonstrate the superiority of our proposed approach when compared with other state-of-the-art trackers. project homepage http://bmal.hust.edu.cn/project/KMF2JMTtracking.html

Robust Correlation Tracking for UAV Videos via Feature Fusion and Saliency Proposals

Following the growing availability of low-cost, commercially available unmanned aerial vehicles (UAVs), more and more research efforts have been focusing on object tracking using videos recorded from UAVs. However, tracking from UAV videos poses many challenges due to platform motion, including background clutter, occlusion, and illumination variation. This paper tackles these challenges by proposing a correlation filter-based tracker with feature fusion and saliency proposals. First, we integrate multiple feature types such as dimensionality-reduced color name (CN) and histograms of oriented gradient (HOG) features to improve the performance of correlation filters for UAV videos. Yet, a fused feature acting as a multivector descriptor cannot be directly used in prior correlation filters. Therefore, a fused feature correlation filter is proposed that can directly convolve with a multivector descriptor, in order to obtain a single-channel response that indicates the location of an object. Furthermore, we introduce saliency proposals as re-detector to reduce background interference caused by occlusion or any distracter. Finally, an adaptive template-update strategy according to saliency information is utilized to alleviate possible model drifts. Systematic comparative evaluations performed on two popular UAV datasets show the effectiveness of the proposed approach.

Online adaptive complementation tracker

Correlation filter-based trackers have recently shown excellent performance in terms of motion blur and illumination changes, but they are notoriously sensitive to deformation. It has been demonstrated that the combination of the correlation filter-based tracker and the color histogram-based tracker can alleviate the deformation and keep advantages of the correlation filter-based tracker. However, the most existing complementary tracking algorithms, which use fixed complementary weights, limit the performance of every sub tracker. This paper introduces an adaptive complementary tracker by online learning dynamic complementary weights. The strategy enables inappropriate sub tracker to be down-weighted while increasing the impact of suitable one. We jointly learn the sub trackers and their reliability weights by regression analysis of the corresponding historical tracking results. The robustness of the model also can be improved by training each sub tracker with the result of historical tracking. Finally, both qualitative and quantitative evaluations demonstrate that our tracker achieves the state-of-the-art results in a wide range of tracking scenarios.

Support vector correlation filter with long-term tracking

Boosted by the promising advancement of the correlation filter-based tracker, we propose an algorithm called the SLT (support vector correlation filter with long-term tracking) that is based on the new SCF (support vector correlation filter) framework to handle long-term tracking. To perform long-term tracking, we propose using a detector to refine the position that includes occlusion and deformation and is out-of-view. We used a new judgment criterion called the max response to the average response rate (MAR) to activate the re-detection procedure and then exploit the linear support vector machine (SVM) classifier to obtain a positive refinement. Moreover, we do not update the SVM classifier every frame to reduce the number of computations and obtain better samples to improve the accuracy of the classifier. We use the online passive–aggressive learning algorithm for online learning and use the same MAR criterion to active it. Extensive experimental results on the OTB50 benchmark dataset show its superior performance in terms of accuracy and robustness.

Kernel correlation filters for visual tracking with adaptive fusion of heterogeneous cues

Abstract Although the correlation filter-based trackers have achieved competitive results both on accuracy and robustness, the performance of trackers can still be improved because the most existing trackers either use a fixed scale or a sole filtering template to represent a target object. In this paper, to effectively handle the scale variation and the drifting problem, we propose a correlation filter-based tracker by adaptively fusing the heterogeneous cues. Firstly, to tackle the problems of the fixed template size, the scale of a target object is estimated from a set of possible scales. Secondly, an adaptive set of filtering templates is learned to alleviate the drifting problem by carefully selecting object candidates in different situations to jointly capture the target appearance variations. Finally, a variety of simple yet effective features (e.g., the HOG and color name features) are effectively integrated into the learning process of filters to further improve the discriminative power of the filters. Consequently, the proposed correlation filter-based tracker can simultaneous utilizes different types of cues to effectively estimate the target's location and scale while alleviating the drifting problem. We have done extensive experiments on the CVPR2013 tracking benchmark dataset with 50 challenging sequences. The proposed tracker successfully tracked the targets in about 90% videos and outperformed the state-of-the-art trackers.

Object Tracking in Vary Lighting Conditions for Fog Based Intelligent Surveillance of Public Spaces

With rapid development of computer vision and artificial intelligence, cities are becoming more and more intelligent. Recently, since intelligent surveillance was applied in all kind of smart city services, object tracking attracted more attention. However, two serious problems blocked the development of visual tracking in real applications. The first problem is its lower performance under intense illumination variation, while the second issue is its slow speed. This paper addressed these two problems by proposing a correlation filter-based tracker. Fog computing platform was deployed to accelerate the proposed tracking approach. The tracker was constructed by multiple positions’ detections and alternate templates. The detection position was repositioned according to the estimated speed of target by an optical flow method, and the alternate template was stored with a template update mechanism, which is all computed at the edge. Experimental results on large-scale public benchmark data sets showed the effectiveness of the proposed method in comparison with state-of-the-art methods.