Robust Appearance Model Research Articles

Learning dynamic spatial-temporal regularized correlation filter tracking with response deviation suppression via multi-feature fusion

Visual object tracking (VOT) for intelligent video surveillance has attracted great attention in the current research community, thanks to advances in computer vision and camera technology. Meanwhile, discriminative correlation filter (DCF) trackers garnered significant interest owing to their high accuracy and low computing cost. Many researchers have introduced spatial and temporal regularization into the DCF framework to achieve a more robust appearance model and further improve tracking performance. However, these algorithms typically set fixed spatial and temporal regularization parameters, which limit flexibility and adaptability under cluttered and challenging scenarios. To overcome these problems, in this work, we propose a new dynamic spatial–temporal regularization for the DCF tracking model that emphasizes the filter to concentrate on more reliable regions during the training stage. Furthermore, we present a response deviation-suppressed regularization term for responses to encourage temporal consistency and avoid model degradation by suppressing relative response changes between two consecutive frames. Moreover, we introduce a multi-memory tracking framework to exploit various features and each memory contributes to tracking the target across all frames. Significant experiments on the OTB-2013, OTB-2015, TC-128, UAV-123, UAVDT, and DTB-70 datasets have revealed that the performance thereof outperformed many state-of-the-art trackers based on DCF and deep-based frameworks in terms of tracking accuracy and tracking success rate.

Learning modality feature fusion via transformer for RGBT-tracking

RGB-T tracking can be seen as multi-view fusion tracking, and in this study, we propose a network with transformer structure, Multi-Modal Mutual Propagation Tracker (MMMPT). In order to obtain robust appearance model from multi-modal data, we adopt encoder–decoder architecture for extract information. In the encoding stage, the template features of multiple frames enhance the common features across them through the self-attention mechanism to obtain time-invariant target representation. At the same time, it also interacts with multi-modal data through cross-modal propagation, resulting in a modal-invariant representation of the target. The transformer decoder transfers useful information from the template to search areas through a similarity matrix. We experiment on the RGBT234, GTOT, VTUAV and LasHeR datasets to assess the RGBT-transformer tracker. Extensive experiments indicate that our proposed framework is not inferior to the state-of-the-art trackers in terms of robustness and accuracy.

Learning Adaptive Spatial Regularization and Temporal-Aware Correlation Filters for Visual Object Tracking

Recently, discriminative correlation filters (DCF) based trackers have gained much attention and obtained remarkable achievements for their high efficiency and outstanding performance. However, undesirable boundary effects occur when the DCF-based trackers suffer from challenging situations, such as occlusion, background clutters, fast motion, and so on. To address these problems, this work proposes a novel adaptive spatial regularization and temporal-aware correlation filters (ASTCF) model to deal with the boundary effects which occur in the correlation filters tracking. Firstly, our ASTCF model learns a more robust correlation filter template by introducing spatial regularization and temporal-aware components into the objective function. The adaptive spatial regularization provides a more robust appearance model to handle the large appearance changes at different times; meanwhile, the temporal-aware constraint can enhance the time continuity and consistency of this model. They make correlation filters model more discriminating, and also reduce the influence of the boundary effects during the tracking process. Secondly, the objective function can be transformed into three sub-problems with closed-form solutions and effectively solved via the alternating direction method of multipliers (ADMM). Finally, we compare our tracker with some representative methods and evaluate using three different benchmarks, including OTB2015, VOT2018 and LaSOT datasets, where the experimental results demonstrate the superiority of our tracker on most of the performance criteria compared with the existing trackers.

Mutation detection dual correlation filter with an object-awareness module for real-time target tracking.

Discriminative correlation filter (DCF) based methods have recently been widely used for visual tracking tasks. The adaptive spatiotemporal-regulation based tracker (AutoTrack) can only partially solve some limitations of the DCF framework including filter degradation and the boundary effect, but its application scenarios need to be broadened, and performance improvements are also required. To further surmount these difficulties, this paper provides an object-awareness-module based mutation detection dual correlation filter (MDDCF-OAM). The main innovation points of this work are: (1) an object-mask based context enhancer is proposed to formulate a more robust appearance model; (2) a dual filter training-learning structure is adopted to allow the dual filters to restrict each other and suppress the filter degradation effect; (3) a Gaussian label map is updated with the refined joint response map to detect and attenuate the response mutation effects. Exhaustive experiments have been conducted to test the efficiency of the suggested MDDCF-OAM on four benchmarks, namely, OTB2015, UAV123, TC128, and VOT2019. The results indicate that: (1) the introduced MDDCF-OAM surpasses nine state-of-the-art trackers; (2) the MDDCF-OAM has a real-time speed of 32 frames per second, which is sufficient for target tracking tasks in numerous scenarios, especially unmanned aerial vehicles and camera tracking.

Multi‐Traffic Targets Tracking Based on an Improved Structural Sparse Representation with Spatial‐Temporal Constraint

Vehicles or pedestrians tracking is an important task in intelligent transportation system. In this paper, we propose an online multi-object tracking for intelligent traffic platform that employs improved sparse representation and structural constraint. We first build the spatial-temporal constraint via the geometric relations and appearance of tracked objects, then we construct a robust appearance model by incorporating the discriminative sparse representation with weight constraint and local sparse appearance with occlusion analysis. Finally, we complete data association by using maximum a posteriori in a Bayesian framework in the pursuit for the optimal detection estimation. Experimental results in two challenging vehicle tracking benchmark datasets show that the proposed method has a good tracking performance.

Using Segmentation With Multi-Scale Selective Kernel for Visual Object Tracking

Generic visual object tracking is challenging due to various difficulties, e.g. scale variations and deformations. To solve those problems, we propose a novel multi-scale selective kernel module for tracking, which contains small-scale and large-scale branches to model the target at different scales and attention mechanism to capture the more effective appearance information of the target. In our module, we cascade multiple small-scale convolutional blocks as an equivalent large-scale branch to extract large-scale features of the target effectively. Besides, we present a hybrid strategy for feature selection to extract significant information from features of different scales. Based on the current excellent segmentation tracking framework, we propose a novel tracking network that leverages our module at multiple places in the up-sample phase to construct a more accurate and robust appearance model. Extensive experimental results show that our tracker outperforms other state-of-the-art trackers on multiple challenging benchmarks including VOT2018, TrackingNet, DAVIS-2017, and YouTube-VOS-2018 while achieves real-time tracking.

Disruptor-Aware Interval-Based Response Inconsistency for Correlation Filters in Real-Time Aerial Tracking

Aerial object tracking approaches based on discriminative correlation filter (DCF) have attracted wide attention in the tracking community due to their impressive progress recently. Many studies introduce temporal regularization into the DCF-based framework to achieve a more robust appearance model and further enhance the tracking performance. However, existing temporal regularization approaches usually utilize the information of two consecutive frames, which are not robust enough due to limited information. Although some methods attempt to incorporate abundant training samples and generally improve the tracking performance, these improvements are at the expense of significantly increased computing consumption. Besides, most existing methods introduce historical information directly without denoising, which means that background noises are also introduced into the filter training and may degrade the tracking accuracy. To tackle the drawbacks mentioned earlier, this work proposes a novel aerial object tracking approach to exploit disruptor-aware interval-based response inconsistency, i.e., IBRI tracker. The proposed method is able to incorporate historical interval information by utilizing responses in the filter training process, thereby obtaining a robust tracking performance while maintaining the real-time speed. Moreover, to reduce the disruptions caused by similar object, partial occlusion, and other challenging scenes, a novel disruptor-aware scheme based on response bucketing is introduced to detect the disruptor and enforce a spatial penalty for the disruptive area around the tracked object. Exhausted experiments on multiple well-known challenging aerial tracking benchmarks demonstrate the accuracy and robustness of the proposed IBRI tracker against other 35 state-of-the-art trackers. With a real-time speed of ~32 frames/s on a single CPU, the proposed approach can be applied for typical aerial platforms to achieve aerial visual object tracking efficiently.

Aberrance suppressed spatio-temporal correlation filters for visual object tracking

The objective of the present study is to design a correlation filter-based tracking method for robust visual object tracking. In the literature, numerous tracking methods have been proposed based on discriminative correlation filter (DCF) and obtained impressive performance. However, existing algorithms still face difficulties such as partial occlusion, clutter background, uncertainties, boundary effects (especially when the target search area is small) and other challenging visual factors. Furthermore, during the target detection process, the sudden changes in objects caused by illumination variations and partial/full occlusion degrade the performance. To tackle the drawbacks mentioned earlier, we propose a tracking algorithm concerning the aberrance suppressed correlation filters with spatio-temporal information for visual tracking. Specifically, we introduce a spatial regularization term into the correlation filter to suppresses the boundary effects. Following that, a temporal regularization is adopted into the DCF-based framework to achieve a more robust appearance model and further enhance the tracking performance. In addition, we introduce an approach to suppress the aberrance in response maps caused by the sudden changes. Technically, our proposed method can be directly solved by using the alternating direction method of multipliers (ADMM) technique with a low computational cost. Finally, extensive experimental results on OTB2013, OTB2015, TempleColor128 and UAV123 datasets demonstrate that the proposed method performs favorably against state-of-the-art methods.

Robust Visual Tracking Based on Convolutional Sparse Coding

This paper proposes a new visual tracking method by constructing the robust appearance model of the target with convolutional sparse coding. First, our method uses convolutional sparse coding to divide the interest region of the target into a smooth image and four detail images with different fitting degrees. Second, we compute the initial target region by tracking the smooth image with the kernel correlation filtering. We define an appearance model to describe the details of the target based on the initial target region and the combination of four detail images. Third, we propose a matching method by the overlap rate and Euclidean distance to evaluate candidates and the appearance model to compute the tracking results based on detail images. Finally, the two tracking results are separately computed by the smooth image, and the detail images are combined to produce the final target rectangle. Many experiments on videos from Tracking Benchmark 2015 demonstrate that our method produces much better results than most of the present visual tracking methods.

Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking.

In this study, we propose a novel RGB-T tracking framework by jointly modeling both appearance and motion cues. First, to obtain a robust appearance model, we develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. The fusion weights are determined by using offline-trained global and local multimodal fusion networks, and then adopted to linearly combine the response maps of RGB and T modalities. Second, when the appearance cue is unreliable, we comprehensively take motion cues, i.e., target and camera motions, into account to make the tracker robust. We further propose a tracker switcher to switch the appearance and motion trackers flexibly. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.

Learning Dynamic Spatial-Temporal Regularization for UAV Object Tracking

With the wide vision and high flexibility, unmanned aerial vehicle (UAV) has been widely used into object tracking in recent years. However, its limited computing capability poses a great challenges to tracking algorithms. On the other hand, Discriminative Correlation Filter (DCF) based trackers have attracted great attention due to their computational efficiency and superior accuracy. Many studies introduce spatial and temporal regularization into the DCF framework to achieve a more robust appearance model and further enhance the tracking performance. However, such algorithms generally set fixed spatial or temporal regularization parameters, which lack flexibility and adaptability under cluttered and challenging scenarios. To tackle such issue, in this letter, we propose a novel DCF tracking model by introducing dynamic spatial regularization weight, which encourage the filter focuses on more reliable region during training stage. Furthermore, our method could optimize the spatial and temporal regularization weight simultaneously using Alternative Direction Method of Multiplies (ADMM) technique method, where each sub-problem has closed-form solution. Through the joint optimization, our tracker could not only suppress the potential distractors but also construct robust target appearance on the basis of reliable historical information. Experiments on two UAV benchmarks have demonstrated that our tracker performs favorably against other state-of-the-art algorithms.

Learning Target‐Adaptive Correlation Filters for Visual Tracking

AbstractCorrelation filters (CF) achieve excellent performance in visual tracking but suffer from undesired boundary effects. A significant amount of approaches focus on enlarging search regions to make up for this shortcoming. However, this introduces excessive background noises and misleads the filter into learning from the ambiguous information. In this paper, we propose a novel target‐adaptive correlation filter (TACF) that incorporates context and spatial‐temporal regularizations into the CF framework, thus learning a more robust appearance model in the case of large appearance variations. Besides, it can be effectively optimized via the alternating direction method of multipliers(ADMM), thus achieving a global optimal solution. Finally, an adaptive updating strategy is presented to discriminate the unreliable samples and alleviate the contamination of these training samples. Extensive evaluations on OTB‐2013, OTB‐2015, VOT‐2016, VOT‐2017 and TC‐128 datasets demonstrate that our TACF is very promising for various challenging scenarios compared with several state‐of‐the‐art trackers, with real‐time performance of 20 frames per second(fps).

Near-Online Multi-Pedestrian Tracking via Combining Multiple Consistent Appearance Cues

An important cue for multi-pedestrian tracking in video is the consistent appearance of an individual for quite a while. In this paper, we address multi-pedestrian tracking by learning a robust appearance model from the paradigm of tracking by detection. To separate detections of different pedestrians while assembling detections of the same pedestrian, we take advantage of the cue of consistent appearance and exploit three types of evidence from the recent, past and near-future. Existing online approaches only exploit the detection-to-detection and sequence-to-detection metrics, which focus on the recent and past appearance patterns respectively, while the future pedestrian appearance is simply ignored. This drawback is remedied in this paper by further considering the sequence-to-sequence metric, which resorts to near-future appearance presentation. Adaptive combination weights are learned to fuse these three different metrics. Moreover, we propose a novel Focal Triplet Loss to make the model focus more on hard examples than the easy ones. We demonstrate that this can significantly enhance the discriminating power of the model compared with treating every sample equally. Effectiveness and efficiency of the proposed method is verified by conducting comprehensive ablation studies and comparing with many competitive (offline/online/near-online) counterparts on the MOT16 and MOT17 Challenges.

A novel approach for multi-cue feature fusion for robust object tracking

Object tracking is a significant problem of computer vision due to challenging environmental variations. Single cue appearance model is not sufficient to handle the variations. To this end, we propose a multi-cue tracking framework in which complementary cues namely, LBP and HOG were exploited to develop a robust appearance model. The proposed feature fusion captures the high-level relationship between the features and diminishes the low-level relationship. Transductive reliability is also integrated at each frame to make tracker adaptive with the changing environment. In addition, K-Means based classifier creates clear and concise boundary between positive and negative fragments which are further used to update the reference dictionary. This adaptation strategy prevents the erroneous updation of the proposed tracker during background clutters, occlusion, and fast motion. Qualitative and quantitative analysis on challenging video sequences from OTB-100 dataset, VOT dataset and UAV123 reveal that the proposed tracker performs favorably against 13 others state-of-the-art trackers.

Saliency Guided Visual Tracking via Correlation Filter With Log-Gabor Filter

Correlation filter (CF) based tracking algorithms have tremendously contributed to the field of visual tracking due to the high computational efficiency and competitive performance. Nonetheless, most CF-based trackers are vulnerable to the influence of occlusion and boundary effect, which results in suboptimal performance. In this article, we propose saliency guided visual tracking via correlation filter with log-Gabor filter to robustify its performance under occlusion and boundary effect challenges. Firstly, we propose the CF with log-Gabor filter to get a robust appearance model. The log-Gabor filter is adopted to preprocess the sequence to gain the log-Gabor feature, which provides important cues for tracking since it encodes the texture information. Secondly, considering the prior information, we embed the novel saliency guided adaptive spatial feature selection to filter learning to preserve the spatial structure in the lower manifold and mitigate boundary distortion. Thirdly, the occlusion estimating strategy, performing on-line evaluation of tracking, triggers the motion estimation module to optimize the optimal location. Experiments on benchmark databases demonstrate the enhanced discrimination and interpretability of the proposed tracker and its superiority over other trackers.

Visual tracking based on robust appearance model

This paper aims at solving the problem of target drifting or losing due to the dramatic changes in the appearance of the target caused by the presence of severe target occlusion, background clutter, and illumination variations in video sequence. Firstly, we obtain a discriminative appearance description of the target by calculating the global and local Fisher vectors of the target area, and subsequently we utilize a semi-supervised linear kernel classifier to calculate the confidence of each candidate for well distinguish the foreground target from the background region. Meanwhile, as the change in the appearance of the object has caused serious pollution and interfered with the original information of the target area, we obtain the similarity between the sub-patches of the target template set and all the candidates, according to the pollution degree of patches of each candidate. Finally, we retrieve the best candidate according to the production of the weight, confidence and the similarity. The extensive experiment results show that our algorithm can effectively cope with the pollution of the target area and the drastic change of the target appearance caused by the severe occlusion and illumination changes in complex scenes. This also demonstrates that our algorithm has considerably accuracy and robustness in visual tracking.

MSSTResNet-TLD: A robust tracking method based on tracking-learning-detection framework by using multi-scale spatio-temporal residual network feature model

The performance of tracking task is directly dependent on the appearance features of target object, a robust approach for constructing appearance features is crucial for adaptation the appearance change. To construct an accurate and robust appearance model for visual object tracking, we modify original deep residual learning network architecture and name it Multi-Scale Residual Network (MSResNet). The first video frame image and its related information of the current input video sequence are used to learn a multi-scale appearance model of target object and a loss function is minimized over the appearance features. Meanwhile, spatial information of each video frame and temporal information between successive video frames effectively combine with MSResNet. And thus the features are generated by Multi-Scale Spatio-Temporal Residual Network, which is named MSSTResNet feature model, can adapt to scale variation, illumination variation, background clutters, severe deformation of the target object, and so on. We implement a robust tracking method based on tracking-learning-detection framework by using our proposed MSSTResNet feature model and name it MSSTResNet-TLD tracker. Unlike the previous tracking methods, the MSResNet architecture is not offline pre-trained on a large auxiliary datasets but is directly learned end-to-end with a multi-task loss by using the current input video sequence. Furthermore, the multi-task loss function utilizes the classification loss and regression loss that is more accurate for target localization. Our experimental results demonstrate that the proposed tracking method outperforms the current state-of-the-art tracking methods on Visual Object Tracking Benchmark (VOT-2016), Object Tracking Benchmark (OTB-2015), and Unmanned Aerial Vehicles (UAV20L) test datasets. Furthermore, our MSSTResNet-TLD tracker is faster than previous most trackers based on deep Convolutional Neural Network (ConvNet or CNN) and our tracker is extremely robust to the tiny target object. Our source code is available for download athttps://github.com/binger1225/MSSTResNet-TLD-Tracker.

STResNet_CF Tracker: The Deep Spatiotemporal Features Learning for Correlation Filter Based Robust Visual Object Tracking

Constructing a robust appearance model of the visual object is a crucial task for visual object tracking. Recently, more and more studies combine spatial feature with a temporal feature to improve the tracking performance. These methods successfully apply the features from spatial and temporal to address the problem for tracking. This paper presents a novel method for visual object tracking based on spatiotemporal feature combined with correlation filters. In this paper, the visual features of a target object are extracted from a spatial–temporal residual network (STResNet) appearance model with two sub-networks. The STResNet appearance model learns separately spatial feature and temporal feature, respectively, so that we can effectively utilize spatial context around the surrounding of the target object in each frame and the temporal relationship between successive frames to refine the appearance representation of the target object. Finally, our spatiotemporal fusion feature from STResNet appearance model is incorporated into the correlation filter for robust visual object tracking. The experimental results show that our method achieves similar or better performance compared with the other tracking methods based on convolutional neural networks or correlation filter.

Visual Tracking Based on Adaptive Mean Shift Multiple Appearance Models

To overcome the tracking issues caused by the complex environment namely, illumination variation and background clutters, tracking algorithm was proposed based on multi-cues fusion to construct a robust appearance model, indeed the global motion is estimated using the H∞ filter based on the nearly constant velocity motion model, then the traditional Mean Shift (MS) estimate the local state associated with each sub appearance model, finally the weights of the sub appearance models are adjusted and combined to estimate the final state. The proposed method is tested on public videos that present different environment issues. Experiences and comparisons conducted show the robustness of our methods in challenging tracking conditions.

Complementary Tracking via Dual Color Clustering and Spatio-Temporal Regularized Correlation Learning

Recently, a simple, yet effective and efficient tracker named Staple has achieved promising performance in terms of efficiency and accuracy on a series of visual tracking benchmarks. Staple is equipped with complementary learners of discriminative correlation filters (DCFs) and color histograms, which are robust to both color changes and deformations. However, it has some drawbacks: 1) Staple only employs standard color histograms with the same quantization step for all sequences, which does not consider the specific structural information of target in each sequence, thereby affecting its discriminative capability to separate target from background. 2) The standard DCFs are efficient but suffer from unwanted boundary effects, leading to failures in some challenging scenarios. To address these issues, we present a dual color clustering and spatio-temporal regularized correlation regressions-based complementary tracker (CSCT). The proposed CSCT includes two components with complementary merits to adaptively deal with significant color variations and deformations for each sequence: First, we design a novel color clustering-based histogram model that first adaptively divides the colors of the target in the 1st frame into several cluster centers, and then the cluster centers are taken as references to construct adaptive color histograms for targets in the coming frames, which enable to adapt significant target deformations. Second, we propose to learn spatio-temporal regularized CFs, which not only enable to avoid boundary effects but also provides a more robust appearance model than the discriminative CFs in Staple in the case of large appearance variations. Compared to Staple, our CSCT with handcrafted features achieves a gain of 5.9%, 3.4%, and 1.5% on OTB100, Temple-Color, and VOT2016 benchmarks in terms of AUC and EAO scores, respectively. Moreover, our CSCT performs favorably against several state-of-the-art trackers, including the deep learning-based trackers.