Correlation Filters For Object Tracking Research Articles

Deep-feature-based asymmetrical background-aware correlation filter for object tracking

Correlation filter-based video object-tracking algorithms have gained widespread attention due to their efficiency and excellent tracking performance. However, traditional correlation filtering tracking algorithms possess several limitations. (1) They extract image features only by using rule sampling, which ignores the shape information of the target, resulting in insufficient discriminative power of the features. (2) They focus only on the difference between the background and the object, ignoring the effects of more challenging intra-class interference. (3) They do not further evaluate the reliability of the optimal candidate samples, resulting in easy failure in occlusion scenarios. This paper proposes an asymmetric background-aware correlation filter method for object tracking with deep features to solve the above limitations. First, an asymmetrical background-aware sampling method based on the shape information of the object is proposed. This sampling method significantly differs from the symmetrical sampling method in the traditional correlation filter framework. By exploring the shape information of the object, improving the otherness between the background and object samples is easy, thus suppressing the intra-class interferences. Second, deep neural networks are introduced in the correlation filter framework to extract the deep object features and a spatio-temporal regularization factor is adaptively assigned to suppress the boundary effects, intra-class distractors and aberrance between frames. Finally, a multi-modal object pool is constructed to evaluate the optimal candidate sample in each frame. This template pool fully exploits object diversity and solves the tracking drift and failure caused by invalid appearance changes in scenes such as occlusion and vigorous motion scenarios. To validate the effectiveness of the proposed method, it was compared with the state-of-the-art methods on public datasets. The experimental results show that the tracking performance of the proposed method is competitive.

Visual Object Tracking Based on Light-Field Imaging in the Presence of Similar Distractors

Visual object tracking is of great importance in the field of computer vision. One of the main challenges is the difficulty of identifying moving targets from nearby similar distractors with a single-view image of the scene. To overcome this challenge, in this article, we acquire multiview images of the scenes by using a light-field camera. The multiview images are able to capture the 4-D structure instead of the 2-D plane of the objects but are more difficult to process. Therefore, we propose a novel representation for multiview images, i.e., the macro-epipolar plane image (macro-EPI), which highlights both spatial topological and angular information of the target and distractors. It is obtained by slicing the original multiview images into pieces and properly restacking these pieces in an ordinal manner. The resulting macro-EPI is mapped into the 2-D space; therefore, we adapt a modified autoencoder network to train a macro-EPI feature extractor. Thereafter, we design a composite framework of two-pattern convolution filters based on a discriminative correlation filter for object tracking, which successfully discriminates the target from the distractors by merging the macro-EPI features and the single-view image features. The experiments also show that our method outperforms the state-of-the-art methods in the presence of similar distractors.

Redefined target sample-based background-aware correlation filters for object tracking

Redefined target sample-based background-aware correlation filters for object tracking

Feature adaptation-based multipeak-redetection spatial-aware correlation filter for object tracking

Visual tracking has always been an important research topic in the field of computer vision. During tracking, cluttered backgrounds, deformations, and occasional occlusion inevitably cause unpredictable appearance changes in the target, making the task very challenging. Traditional methods often tackle this problem using fixed central region constraints, simple combinations of features and independent classifiers. However, existing constraints are not very accurate; the simple mechanism of combining features lacks descriptive robustness, and the independent classifiers require complex additional training. Therefore, in this paper, we propose a correlation filter tracker called the feature adaptation-based multipeak-redetection spatial-aware correlation filter (FMCF) to accurately track targets that often change in appearance. In this tracker, the correlation filter has a dynamic spatial constraint that effectively reduces the response of the filter to the cluttered background area. To address target deformation, a new feature adaptation-based training method is designed, and the final response is a combination of the correlation filter and an independent statistical color model. Finally, a multipeak-redetection scheme, which can find the missing targets within multiple peaks of the final response map, is proposed to handle occlusion. In particular, the statistical color model is employed in the spatial-aware filter construction, the final response calculation, and the multipeak-redetection procedure to improve the robustness of the tracker, because of its ability to generate pixel-level color probabilities. Extensive experimental results on public tracking datasets prove that our proposed tracker can perform better than state-of-the-art trackers. Moreover, compared to the baseline tracker, the precision rate and success rate of FMCF are increased by 6.5% and 4.8% on the OTB100 benchmark and by 12.7% and 8.2% on the TC128 benchmark, respectively.

Co-saliency-regularized correlation filter for object tracking

Correlation filtering (CF) has emerged as one of the best object-tracking frameworks, achieving a good balance between tracking accuracy and speed. Based on this framework, many successful trackers have been developed. We propose a new co-saliency regularization method for visual object tracking based on the correlation filter, called the CRCF. To the best our knowledge, this is the first application of co-saliency regularization to CF-based tracking, in the context of widely used co-saliency detection and regularization methods. In the CRCF, the co-saliency information is extracted and introduced into the regularization component in the relevant tracking framework. The model first calculates the salient regions based on the feature differences between salient objects and background information. A co-saliency map is generated for the global correspondence of implicit learning between multiple images. To avoid traversing all the pixels, we use SLIC superpixel clustering to extract the objects saliency information, thus significantly reducing the computational overhead in the image processing step while retaining the image features and structural information. Finally, during the tracking process, it is possible to dynamically learn the co-saliency map configuration, highlight relevant object areas, and mitigate the impact of the lack of discriminatory representations on the performance. Quantitative and qualitative experimental results on OTB-2015, UAV123, LaSOT, GOT-10k and VOT-2018 datasets show that our CRCF outperforms the latest trackers.

Spatial–Spectral Weighted and Regularized Tensor Sparse Correlation Filter for Object Tracking in Hyperspectral Videos

Hyperspectral video camera captures spatial, spectral and temporal information of moving objects. Traditional object tracking methods developed for color videos have been applied to hyperspectral videos after compressing hundreds of spectral bands into three, which does not fully utilize the wealth spectral information. In order to address this issue, we present a tensor sparse correlation filter with a spatial-spectral weighted regularizer for object tracking. First, tensor processing is employed to reduce the spectral differences of homogeneous background, thereby producing robust spectral structure features. Second, a spatial-spectral weighted regularizer is designed in the correlation filter framework to penalize filter template by suppressing spectral features dissimilar to the center pixel in tracking. Third, a sparse constraint term and tracking context information are incorporated to suppress unexpected peaks in the response map. Finally, a reformulated stacked HOG feature extractor and a two-dimensional adaptive scale search strategy are developed to further improve the tracker’s feature discrimination and scale adaptation capability. Experimental results demonstrate that the proposed method achieves superior tracking performance than traditional correlation filter-based trackers.

Subspace reconstruction based correlation filter for object tracking

The correlation filter (CF) achieves excellent performance, showing high robustness to motion blur or illumination change by learning filters. However, tracking in challenging scenarios with occlusion or out-of-view is still not well resolved. In the scenario of occlusion, the background information is mixed into the image patch to learn the filter, which causes the filter to learn the background. To alleviate this problem, we improve CF trackers by proposing the subspace reconstruction based CF (SRBCF) tracker. In our method, the original image patch for learning filters is replaced by a reconstructed patch when the appearance of the object dramatically changes, such as occlusion or disappearance, so that the filter can learn from the object instead of the background. We construct the subspace with image patches of the searching window in previous frames. To track the changes in the subspace and mitigate the adverse effects of outliers on the subspace during the tracking process, we improve a dynamic L1-PCA method to construct and update the subspace with a slightly extra computational cost. Our method can be embedded in various correlation filter trackers, such as STAPLE and KCF. Extensive experiments on the OTB-100 dataset, UAV123, DTB70, and Temple Color Pure dataset (we removed 49 sequences repeated in OTB-100) validate the effectiveness of our method. The maximum AUC increase reaches 11.2% for the baseline method on DTB70.

Salient region weighted correlation filter for object tracking

Salient region weighted correlation filter for object tracking

Correlation filter for object tracking with temporal-spatial constraint

In this work, inspired by the Passive-Aggressive learning (PA), we proposed a Temporal-Spatial Constraint Correlation Filter (TSCF) model to simultaneously constrain the spatial mask and the update direction of the filter. Firstly, the spatial regular term ensures that the background redundancy information does not interfere with the filter update during the tracking process. Secondly, the temporal regular term ensures that the spatial mask and the filter do not change dramatically. Thirdly, our proposed TSCF model can be effectively solved based on the alternate direction method of multiplier (ADMM), where each sub-problem has a closed solution. Finally, our experiments on the OTB100 benchmark shows that our tracker has efficient performance compare with many advanced algorithms, which get an AUC score of 0.599 and an accuracy of 0.794.

Object tracking with mask-constrained spatial context

Discriminative correlation filters (DCFs) have shown excellent performance in visual tracking. DCF substitutes the sliding windows sampling strategy in traditional tracking methods with circular shift of the context area. Via projecting the filter learning into the frequency domain, DCF achieves satisfying performance and speed. Appropriate context area size has an influence on the performance of correlation filters. Small context area limits the CF’s ability to handle fast motion and partial occlusion, whereas large context area leads the CF to suffer from boundary effect. To make use of a large area of context and alleviate the accompanying drift risk, we propose a mask-constrained context correlation filter for object tracking. We first analyze the traditional window strategy via Taylor series and design a spatial mask that can be covered by a larger context area. Furthermore, the shape of the mask is adaptive to the target variation. Extensive experimental results in OTB-2015, VOT-2014, and VOT-2016 datasets demonstrate that this mask-constrained operation can improve the CF tracker performance in a large margin.

Learning Scale-Adaptive Tight Correlation Filter for Object Tracking.

In this paper, we propose a novel tracking method by formulating tracking as a correlation filtering as well as a ridge regression problem. First, we develop a tight correlation filter-based tracking framework from the signal detection perspective. In this formulation, the correlation filter is set as the same size as the target, which can make full use of the relations of the adjacent image patches and effectively exclude the influence of the background. Specifically, we point out that the novel correlation filter model can be regarded as the ridge regression model which takes into account the different importance of the samples and has the consistent objective with tracking. Second, we focus on the scale variation problem in tracking. By making use of the spatial structure of the correlation filter, the multiscale filter banks can be generated via interpolation to handle the scale estimation problem easily. Third, we present a novel distance importance-based confidence calculation model to determine the final tracking result, which not only makes use of the fine discriminability of the correlation filter but also takes the distance importance of the candidate samples into account to alleviate the impact of similar distractors. Experimental results demonstrate that our method is superior to several state-of-the-art trackers and many other correlation filter-based methods in the benchmark datasets.

Adaptive Feature Fusion Object Tracking with Kernelized Correlation Filters

Aiming at the problem of occlusion in object tracking, adaptive feature fusion object tracking with kernelized correlation filters is proposed to advance the robustness of the correlation filter for object tracking. Firstly, feature fusion based on tracking confidence is proposed to improve the robustness of object tracking. Then combined with the detection method of tracking confidence to solve the object occlusion and tracking failure. Finally, template updating strategy based on tracking confidence is proposed to solve the drift problem. The experiment carries on the OTB100 dataset, and compares with other tracking algorithms. The results show that the tracking precision is 4.5% higher than that of the optimal algorithm, and the tracking success rate is improved by 6.1%. Moreover, the object can be well tracked under the occlusion.

Real-Time Visual Tracking through Fusion Features

Due to their high-speed, correlation filters for object tracking have begun to receive increasing attention. Traditional object trackers based on correlation filters typically use a single type of feature. In this paper, we attempt to integrate multiple feature types to improve the performance, and we propose a new DD-HOG fusion feature that consists of discriminative descriptors (DDs) and histograms of oriented gradients (HOG). However, fusion features as multi-vector descriptors cannot be directly used in prior correlation filters. To overcome this difficulty, we propose a multi-vector correlation filter (MVCF) that can directly convolve with a multi-vector descriptor to obtain a single-channel response that indicates the location of an object. Experiments on the CVPR2013 tracking benchmark with the evaluation of state-of-the-art trackers show the effectiveness and speed of the proposed method. Moreover, we show that our MVCF tracker, which uses the DD-HOG descriptor, outperforms the structure-preserving object tracker (SPOT) in multi-object tracking because of its high-speed and ability to address heavy occlusion.