Target Appearance Model Research Articles

Discriminative target predictor based on temporal-scene attention context enhancement and candidate matching mechanism

Visual tracking is a fundamental task in computer vision, which extracts the target context descriptions and features from the first image frame and tracks the target in subsequent frames with the updated target appearance models and the updated tracking models accordingly. The existing deficiencies of deep learning based visual tracker are: precision-speed dilemma, inadequate context usage and spawning tracking noises. To achieve real-time and accurate tracking with deep learning modules under tracking-by-detection framework, this paper introduced a discriminative target predictor based on temporal-scene attention context enhancement and candidate matching mechanism (ACDP). The contributions of ACDP include: 1) a vision transformer based temporal context enhancer is suggested by formulating the temporal context enhanced feature extractor via multi-head attention mechanism; 2) a target state propagation scheme based scene context enhancer is proposed via constructing the target state matrix and corresponding state forward propagation techniques; 3) a joint prediction method is provided to take full usage of the enhanced context through candidate selection and matching mechanism based on dust-bin conception network; 4) theoretical proofs for vision transformer acceleration scheme and the error bounds under successive tracking failure circumstances are given. In addition, the extensive experiments under OTB100, UAV123, NFS, AVisT and VOT2018 benchmarks evaluate the favorable performances of ACDP and indicate that: 1) the proposed modules in ACDP can achieve comparable tracking results even under difficult sequences and scenarios against other 21 state-of-the-art trackers; 2) the ACDP performs 35 FPS tracking speed on the 5 benchmarks averagely.

Scale-Aware Tracking Method with Appearance Feature Filtering and Inter-Frame Continuity.

Visual object tracking is a fundamental task in computer vision that requires estimating the position and scale of a target object in a video sequence. However, scale variation is a difficult challenge that affects the performance and robustness of many trackers, especially those based on the discriminative correlation filter (DCF). Existing scale estimation methods based on multi-scale features are computationally expensive and degrade the real-time performance of the DCF-based tracker, especially in scenarios with restricted computing power. In this paper, we propose a practical and efficient solution that can handle scale changes without using multi-scale features and can be combined with any DCF-based tracker as a plug-in module. We use color name (CN) features and a salient feature to reduce the target appearance model's dimensionality. We then estimate the target scale based on a Gaussian distribution model and introduce global and local scale consistency assumptions to restore the target's scale. We fuse the tracking results with the DCF-based tracker to obtain the new position and scale of the target. We evaluate our method on the benchmark dataset Temple Color 128 and compare it with some popular trackers. Our method achieves competitive accuracy and robustness while significantly reducing the computational cost.

Target Tracking Based on Radon Transform Data Appearance Modeling

This article mainly focuses on an important challenge in target tracking in complex environments the real-time performance of algorithm operation. A new target appearance model based on Radon transform data is studied, and it is introduced into the correlation filtering framework for filtering template training. A fast tracking algorithm and target scale update scheme based on correlation filtering are proposed. The experimental results show that the tracking algorithm proposed in this paper has better robustness and real-time performance compared to current mainstream tracking algorithms, providing a new technical approach for research related to object detection and tracking. The tracking algorithm proposed in this article can also be seen as a framework, where the projected object can not only be the grayscale of the original pixel, but also include multi-channel color values, HOG, and other attributes.

Hierarchical Spatiotemporal Graph Regularized Discriminative Correlation Filter for Visual Object Tracking.

Visual object tracking is a fundamental and challenging task in many high-level vision and robotics applications. It is typically formulated by estimating the target appearance model between consecutive frames. Discriminative correlation filters (DCFs) and their variants have achieved promising speed and accuracy for visual tracking in many challenging scenarios. However, because of the unwanted boundary effects and lack of geometric constraints, these methods suffer from performance degradation. In the current work, we propose hierarchical spatiotemporal graph-regularized correlation filters for robust object tracking. The target sample is decomposed into a large number of deep channels, which are then used to construct a spatial graph such that each graph node corresponds to a particular target location across all channels. Such a graph effectively captures the spatial structure of the target object. In order to capture the temporal structure of the target object, the information in the deep channels obtained from a temporal window is compressed using the principal component analysis, and then, a temporal graph is constructed such that each graph node corresponds to a particular target location in the temporal dimension. Both spatial and temporal graphs span different subspaces such that the target and the background become linearly separable. The learned correlation filter is constrained to act as an eigenvector of the Laplacian of these spatiotemporal graphs. We propose a novel objective function that incorporates these spatiotemporal constraints into the DCFs framework. We solve the objective function using alternating direction methods of multipliers such that each subproblem has a closed-form solution. We evaluate our proposed algorithm on six challenging benchmark datasets and compare it with 33 existing state-of-the art trackers. Our results demonstrate an excellent performance of the proposed algorithm compared to the existing trackers.

Explaining away results in more robust visual tracking

Many current trackers utilise an appearance model to localise the target object in each frame. However, such approaches often fail when there are similar-looking distractor objects in the surrounding background, meaning that target appearance alone is insufficient for robust tracking. In contrast, humans consider the distractor objects as additional visual cues, in order to infer the position of the target. Inspired by this observation, this paper proposes a novel tracking architecture in which not only is the appearance of the tracked object, but also the appearance of the distractors detected in previous frames, taken into consideration using a form of probabilistic inference known as explaining away. This mechanism increases the robustness of tracking by making it more likely that the target appearance model is matched to the true target, rather than similar-looking regions of the current frame. The proposed method can be combined with many existing trackers. Combining it with SiamFC, DaSiamRPN, Super_DiMP, and ARSuper_DiMP all resulted in an increase in the tracking accuracy compared to that achieved by the underlying tracker alone. When combined with Super_DiMP and ARSuper_DiMP, the resulting trackers produce performance that is competitive with the state of the art on seven popular benchmarks.

Discriminative visual tracking via spatially smooth and steep correlation filters

Recently, many discriminative correlation filter (DCF)-basedapproaches have been developed to alleviate the undesired boundary effects of traditional DCF-based methods. However, these approaches either focus on the target area by suppressing the background information or exploit a spatial regularisation term to penalise filter coefficients far from the target centre, where real negative samples are exceedingly rare and irrelevant information is integrated into filters. Based on the observation that the tracking model is strongly dependent on the quality of training samples, this paper proposes a novel DCF-based tracker to precisely process the training set through a smooth and steeply decreasing function in two respects. On the one hand, the surrounding information is suitably incorporated into the target. On the other hand, every real background patch is collected as a negative sample. In addition, the target appearance model is adaptively updated by achieving an appropriate trade-off between current and historical models according to their reliability. Experimental results show that our method outperforms other state-of-the-art trackers in terms of accuracy and efficiency. In particular, compared with the baseline tracker, our method achieves gains of 7.4% and 9.9% in terms of the area under the curve (AUC) on the OTB-100 and LaSOT datasets, respectively.

Research on Visual Tracking Algorithm Based on Peak Sidelobe Ratio

Aiming at the problems of illumination changes, target deformation and background clutter in the target tracking field, a visual tracking algorithm based on peak sidelobe ratio is proposed. The object-interference model is used to represent the target appearance model, and context information is added to the relevant filtering framework. Training the filter internally to enhance the ability of filter discrimination. In the model update process, it is easier to introduce samples that cannot characterize the target, and use peak sidelobe comparison to update the tracking parameters, which can enhance the generalization ability of the model. Tested with some classic and recently algorithms in the OTB50, OTB100, UAV123, TC128 experimental video data set, the experiment' results show that the visual tracking algorithm that is proposed in the article can track the target more accurately. It has important research on the development of intelligent video surveillance value.

Visual object tracking based on residual network and cascaded correlation filters

Significant progress is made in the field of object tracking recently. Especially, trackers based on deep learning and correlation filters both have achieved excellent performance. However, object tracking still faces some challenging problems such as deformation and illumination. In such kinds of situations, the accuracy and precision of tracking algorithms plunge as a result. It is imminent to find a solution to this situation. In this paper, we propose a tracking algorithm based on features extracted by residual network called Resnet features and cascaded correlation filters to improve precision and accuracy. Firstly, features extracted by a deep residual network trained on other image processing datasets, are robust enough and retain higher resolution, therefore, we exploit Resnet-101 pretrained offline to obtain features extracted by middle and high layers for target appearance model representation. Resnet-101 is deeper compared with other deep neural networks which means it contains more semantic information. Then, the method we propose to combine our correlation filters is superior. We propose cascaded correlation filters generated by handcraft, middle-level and high-level features from residual network to gain better competence. Handcraft features localize target precisely because they contain more spatial details while Resnet features are robust to the target appearance change because they retain more semantic information. Finally, we conduct extensive experiments on OTB2013 and OTB2015 benchmark. The experimental results show that our tracker achieves high performance under all kinds of challenges and performs favorably against other state-of-the-art trackers.

Object-oriented remote sensing image information extraction method based on multi-classifier combination and deep learning algorithm

In recent years, high spatial resolution remote sensing technology has made significant progress. High-resolution remote sensing satellites provide great convenience for high-quality image acquisition. In order to adapt to changes in the appearance of the target, mainstream tracking algorithms often use pattern recognition methods to build a target appearance model with learning capabilities, and use the image frames acquired during the tracking process to update the appearance model. This paper mainly studies the object-oriented remote sensing image information extraction method based on multi-classifier combination and deep learning algorithm. In this paper, we use the splitting mechanism of the tree structure to retain the appearance model with diversity, and through the integrated learning integration strategy, the target position is collaboratively predicted. Through the comparative analysis on the OTB and VOT platforms, the algorithm works well when the requirements of the tracking standards are low (the accuracy threshold is greater than 20 pixels and the success threshold is less than 0.4 pixels). The experimental results in this paper show that compared with other advanced classification methods, the proposed method shows better generalization performance in accuracy, recall, f-measure, g-mean and AUC.

Target detection and tracking based on improved HOG-color feature fusion

In the process of constructing the target appearance model, the traditional HOG feature does not take into account the relationship between adjacent regions when the image is segmented in the feature extraction process, resulting in the regional aliasing effect, and the single feature is inefficient in target detection and tracking. An improved HOG-color feature fusion method for target detection and tracking is proposed. Firstly, HOG feature and color feature were extracted from the target samples respectively. During the HOG feature extraction process, trilinear interpolation was used to eliminate regional aliasing effect. Secondly, Bhattacharyya distance is used to select the HOG feature after interpolation, and the appropriate feature is selected as the HOG feature. Then combine the HOG feature after selection with the color feature; Finally, the filter is obtained by learning the kernel correlation filter, and the image is correlated detected with the filter, and the response output is obtained. The experimental results show that this method is superior to the traditional HOG-color feature fusion target detection and tracking method in both tracking speed and tracking efficiency.

Visual recognition and performance prediction of athletes based on target tracking EIA algorithm

In the past, the research of target tracking was often to track problems in a static background, and the tracking scenes were often stable, and the targets were special. However, target tracking is often a tracking problem in the face of realistic complex scenes, and the target and scene are more complex. Therefore, the target tracking algorithm still faces many challenges in practical applications, especially in sports visual feature recognition. Based on the needs of sports feature recognition, this study combines the EIA algorithm to construct a feature recognition model. Moreover, for the shortcomings of the compressed sensing tracking algorithm that cannot accurately and comprehensively describe the target shape through a single target feature, the multi-feature adaptive fusion method is used to visualize the target appearance model, thus improving the accuracy of target tracking. In addition, this study design experiments to analyze the performance of the algorithm model. The research results show that the algorithm model of this study has certain recognition effects.

Correlation Filter Selection for Visual Tracking Using Reinforcement Learning

Correlation filter has been proven to be an effective tool for a number of approaches in visual tracking, particularly for seeking a good balance between tracking accuracy and speed. However, correlation filter-based models are susceptible to wrong updates stemming from inaccurate tracking results. To date, very little effort has been devoted towards handling the correlation filter update problem. In this paper, we propose a novel approach to address the correlation filter update problem. In our approach, we update and maintain multiple correlation filter models in parallel, and we use deep reinforcement learning for the selection of an optimal correlation filter model among them. To facilitate the decision process in an efficient manner, we propose a decision-net to deal with target appearance modeling, which is trained through hundreds of challenging videos using proximal policy optimization and a lightweight learning network. An exhaustive evaluation of the proposed approach on the OTB100 and OTB2013 benchmarks shows that the approach is effective enough to achieve the average success rate of 62.3% and the average precision score of 81.2%, both exceeding the performance of traditional correlation filter-based trackers.

Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking in surgical video.

Vision-based tracking in an important component for building computer assisted interventions in minimally invasive surgery as it facilitates estimation of motion for instruments and anatomical targets. Tracking-by-detection algorithms are widely used for visual tracking, where the problem is treated as a classification task and a tracking target appearance model is updated over time using online learning. In challenging conditions, like surgical scenes, where tracking targets deform and vary in scale, the update step is prone to include background information in model appearance or to lack the ability to estimate change of scale, which degrades the performance of classifier. In this paper, we propose a Patch-based Adaptive Weighting with Segmentation and Scale (PAWSS) tracking framework that tackles both scale and background problems. A simple but effective colour-based segmentation model is used to suppress background information and multi-scale samples are extracted to enrich the training pool, which allows the tracker to handle both incremental and abrupt scale variations between frames. Experimentally, we evaluate our approach on Online Tracking Benchmark (OTB) dataset and Visual Object Tracking (VOT) challenge datasets, showing that our approach outperforms recent state-of-the-art trackers, and it especially improves successful rate score on OTB dataset, while on VOT datasets, PAWSS ranks among the top trackers while operating at real-time frame rates. Focusing on the application of PAWSS to surgical scenes, we evaluate on MICCAI 2015 challenge instrument tracking challenge and in vivo datasets, showing that our approach performs the best among all submitted methods and also has promising performance on in vivo surgical instrument tracking.

Robust Visual Tracking Based on Relaxed Target Representation

Developing an effective target appearance model is a challenging task in visual tracking under the influences of complicated appearance variations. Many tracking algorithms use a linear combination of previous tracking results to represent a target candidate. In existing target representations, all of the feature elements of a target candidate have the same coding vector. With such type of target representations, robust tracking is not satisfactory when drastic appearance variations occur. In this work, we present a novel appearance model for visual tracking. The proposed appearance model considers the similarity and the distinctiveness of the feature elements of a target candidate. The feature elements should share some similarity to jointly represent a target pattern. We exploit the distinctiveness of feature elements to represent the different importance by introducing a weighted regularization term in the appearance model. A more stable and discriminative target representation is obtained. Superior performance on challenging sequences against state-of-the-art trackers show the robustness of the novel appearance model and the proposed tracker.

Unifying Temporal Context and Multi-Feature With Update-Pacing Framework for Visual Tracking

Model drifting is one of the knotty problems that seriously restricts the accuracy of discriminative trackers in visual tracking. Most existing works usually focus on improving the robustness of the target appearance model. However, they are prone to suffer from model drifting due to the inappropriate model updates during the tracking-by-detection. In this paper, we propose a novel update-pacing framework to suppress the occurrence of model drifting in visual tracking. Specifically, the proposed framework first initializes an ensemble of trackers, each of which updates the model in a different update interval. Once the forward tracking trajectory of each tracker is determined, the backward trajectory will also be generated by the current model to measure the difference with the forward one, and the tracker with the smallest deviation score will be selected as the most robust tracker for the remaining tracking. By performing such self-examination on trajectory pairs, the framework can effectively preserve the temporal context consistency of sequential frames to avoid learning corrupted information. To further improve the performance of the proposed method, a multi-feature extension framework is also proposed to incorporate multiple features into the ensemble of the trackers. The extensive experimental results obtained on large-scale object tracking benchmarks demonstrate that the proposed framework significantly increases the accuracy and robustness of the underlying base trackers, such as DSST, Struck, KCF, and CT, and achieves superior performance compared with the state-of-the-art methods without using deep models.

A template consensus method for visual tracking

Visual tracking is a challenging problem in computer vision. Recently, correlation filter-based trackers have shown to provide excellent tracking performance. Inspired by a sample consensus approach proposed for foreground detection, which classifies a given pixel as foreground or background based on its similarity to recently observed samples, we present a template consensus tracker based on the kernelized correlation filter (KCF). Instead of keeping only one target appearance model in the KCF, we make a feature pool to keep several target appearance models in our method and predict the new target position by searching for the location of the maximal value of the response maps. Both quantitative and qualitative evaluations are performed on the CVPR2013 tracking benchmark dataset. The results show that our proposed method improves the original KCF tracker by 8.17% in the success plot and 8.11% in the precision plot.

Adaptive Discriminative Deep Correlation Filter for Visual Object Tracking

Correlation filter trackers building on deep convolution neural networks (CNNs) contribute efficient visual object trackers but remain challenged with severe target appearance variations. The reason for this is that CNNs trained for image classification tasks are less discriminative to the dynamic variations of targets and backgrounds. In this paper, we propose an adaptive discriminative deep correlation filter (adaDDCF), which, by incorporating discriminative feature fine-tuning with adaptive appearance modeling, pursues stable object tracking in complex backgrounds. In adaDDCF, a convolutional Fisher discriminative analysis (FDA) layer is implemented for positive and negative instance mining and scene-specific feature learning. A correlation layer is then embedded to learn the correlation response of consecutive frames for target appearance modeling. With an online learning procedure using forward–backward propagation, the FDA layer and the correlation layer are effectively coupled, leading to effective and discriminative fine-tuning for the proposed tracker, which consequently alleviates the target drifting problem. Extensive experiments on the challenging benchmarks OTB2013, OTB2015, and OTB50 demonstrate that the proposed adaDDCF tracker outperforms many state-of-the-art trackers.

A Fast Object Tracker Based on Integrated Multiple Features and Dynamic Learning Rate

Object tracking is a vital topic in computer vision. Although tracking algorithms have gained great development in recent years, its robustness and accuracy still need to be improved. In this paper, to overcome single feature with poor representation ability in a complex image sequence, we put forward a multifeature integration framework, including the gray features, Histogram of Gradient (HOG), color-naming (CN), and Illumination Invariant Features (IIF), which effectively improve the robustness of object tracking. In addition, we propose a model updating strategy and introduce a skewness to measure the confidence degree of tracking result. Unlike previous tracking algorithms, we judge the relationship of skewness values between two adjacent frames to decide the updating of target appearance model to use a dynamic learning rate. This way makes our tracker further improve the robustness of tracking and effectively prevents the target drifting caused by occlusion and deformation. Extensive experiments on large-scale benchmark containing 50 image sequences show that our tracker is better than most existing excellent trackers in tracking performance and can run at average speed over 43 fps.

Hierarchical Tracking by Reinforcement Learning based Searching and Coarse-to-fine Verifying.

A class-agnostic tracker typically consists of three key components, i.e., its motion model, its target appearance model, and its updating strategy. However, most recent topperforming trackers mainly focus on constructing complicated appearance models and updating strategies, while using comparatively simple and heuristic motion models that may result in an inefficient search and degrade the tracking performance. To address this issue, we propose a hierarchical tracker that learns to move and track based on the combination of data-driven search at the coarse level, and coarse-to-fine verification at the fine level. At the coarse level, a data-driven motion model learned from deep recurrent reinforcement learning provides our tracker with coarse localization of an object. By formulating motion search as an action-decision problem in reinforcement learning, our tracker utilizes a recurrent convolutional neural network based deep Q-network to effectively learn data-driven searching policies. The learned motion model cannot only significantly reduce the search space, but also provide more reliable interested regions for further verifying. At the fine level, a kernelized correlation filter (KCF) based appearance model is adopted to densely yet efficiently verify a local region centered on the predicted location from the motion model. Through using of circulant matrices and fast Fourier transformation, a large number of candidate samples in the local region can be efficiently and effectively evaluated by the KCF based appearance model. Finally, a simple yet robust estimator is designed to analyze possible tracking failure. The experiments on OTB50 and OTB100 illustrate that our tracker achieves better performance than the state-of-the-art trackers.

Robust Visual Tracking via Dirac-Weighted Cascading Correlation Filters

Correlation filter-based trackers (CFTs) have recently raised considerable attention in visual tracking and achieved competitive performance. Nevertheless, conventional structures of such CFTs build a shallow architecture with a single correlation filter, which cannot comprehensively depict the target appearance. Hence, these trackers lack strong discriminative ability and easily drift when the target suffers drastic appearance variations. To address the limitations, we propose Dirac-weighted cascading correlation filters (DWCCF) for visual tracking. It incorporates cascading characteristics of multiple filters to construct the target appearance model, and dynamically learns Dirac weights for each filter, which is accordingly robust to appearance variations. Besides, we design a boundary penalization strategy to adaptively reduce the boundary effects, which efficiently improves the detection precision for tracking. Qualitative and quantitative evaluations on OTB-2013 and OTB-2015 datasets demonstrate that the proposed DWCCF significantly outperforms other state-of-the-art methods.