Robust Visual Tracking Research Articles

Masked and dynamic Siamese network for robust visual tracking

Visual object tracking is as a critical function for many computer vision tasks such as motion analysis, event detection and action recognition. Recently, Siamese network based trackers gained enormous popularity in the tracking field due to their favorable accuracy and efficiency. However, the distraction problem caused by semantic backgrounds and the simple modeling strategy of target templates often lead to performance degradation. In this study, we propose two modules, namely the target objectness model and the target template model, based on existing Siamese network based trackers to solve these issues. The target objectness model computes the possibility of each pixel in the search area pertaining to the tracked target based on color distributions of the foreground and background areas. The computed target likelihood map is masked on the previous response map, and subsequently adjusts the final response map to focus on the target. This practice enlarges the discrimination between the tracked target and surrounding backgrounds, thus alleviating the distraction problem. The target template model proposes a Gaussian mixed model to encode target appearance variations, where each component of the model represents a different aspect of the target, and the component weights are learned and dynamically updated. The proposed Gaussian model enhances diversity and simultaneously reduces redundancy between target samples. To validate the effectiveness of our proposed method, we perform extensive experiments on four widely used benchmarks, namely OTB100, VOT2016, TC128, and UAV123. The experimental results demonstrate that our proposed algorithm achieves favorable performance compared to many state-of-the-art trackers while maintaining real-time tracking speed.

Robust Visual Tracking Based on Spatial Reliability Constraint

Because of the problem that the target is prone to drift in complex background, a robust tracking algorithm based on spatial reliability constraint is proposed. Firstly, the pre-trained Convolutional Neural Network (CNN) model is used to extract the multi-layer deep features of the target, and the correlation filters are respectively trained on each layer to perform weighted fusion of the obtained response maps. Then, the reliability region information of the target is extracted through the high-level feature map, a binary matrix is obtained. Finally, the obtained binary matrix is used to constrain the search area of the response map, and the maximum response value in the area is the target position. In addition, in order to deal with the long-term occlusion problem, a random selection model update strategy with the first frame template information is proposed. The experimental results show that the proposed algorithm has good performance in dealing with similar background interference, occlusion, and other scenes.

Robust Visual Tracking Based on Relaxed Target Representation

Developing an effective target appearance model is a challenging task in visual tracking under the influences of complicated appearance variations. Many tracking algorithms use a linear combination of previous tracking results to represent a target candidate. In existing target representations, all of the feature elements of a target candidate have the same coding vector. With such type of target representations, robust tracking is not satisfactory when drastic appearance variations occur. In this work, we present a novel appearance model for visual tracking. The proposed appearance model considers the similarity and the distinctiveness of the feature elements of a target candidate. The feature elements should share some similarity to jointly represent a target pattern. We exploit the distinctiveness of feature elements to represent the different importance by introducing a weighted regularization term in the appearance model. A more stable and discriminative target representation is obtained. Superior performance on challenging sequences against state-of-the-art trackers show the robustness of the novel appearance model and the proposed tracker.

Robust visual tracking based on response stability

In this paper, a new approach of response stability based for visual object tracking is developed. This approach proposes a response stability criterion to measure the tracking quality and fuse tracking results of multiple layers of a convolutional neural network (CNN). Inspired by recent detection based methods for visual tracking, the detection capability of EdgeBoxes is investigated, and proposes to re-detect target when tracking failure occurs. In addition, 3D locally adaptive regression kernels (LARK) feature is employed in correlation filter based tracking framework. Extensive experimental results and performance compared with state-of-the-art tracking algorithms on challenging benchmark datasets show that our method is more accurate and robust.

Robust adaptive visual tracking control for uncertain robotic systems with unknown dead-zone inputs

This paper is concerned with the image-based visual servoing (IBVS) control for uncalibrated camera-robot system with unknown dead-zone constraint, where the uncertain kinematics and dynamics are also considered. The control implementation is achieved by constructing a smooth inverse model for dead-zone-input to eliminate the nonlinear effect resulting from the actuator constraint. A novel adaptive algorithm, which does not require a priori knowledge of the parameter intervals of dead-zone model, is proposed to update the parameter values online, and the dead-zone slopes are not required the same. Furthermore, to accommodate the uncertainties of uncalibrated camera-robot system, adaptation laws are developed to estimate the uncertain parameters, simultaneously avoiding singularity of the image Jacobian matrix. With the full consideration of unknown dead-zone constraint and system uncertainties, an adaptive robust visual tracking control scheme together with dead-zone compensation is subsequently established such that the image tracking error converges to the origin. Based on a 3-DOF manipulator, simulations are conducted to verify the tracking performance of the proposed controller.

Real-time robust complementary visual tracking with redetection scheme

Recently, correlation filter-based trackers have been widely investigated due to their high efficiency and robustness. However, most of them use a fixed cosine window to deal with boundary effects and ignore the reliability of tracking results, which results in poor tracking performance when the target endures severe occlusion and large appearance variation. In order to deal with these issues, we propose a tracking framework with an adaptive cosine window, which is composed of a reliability estimation module and a redetection module. First, we incorporate the object likelihood map into the traditional fixed cosine window to form an adaptive cosine window, which can enlarge the searching region and effectively cope with boundary effects. Second, the peak-to-sidelobe ratio of HOG-based correlation response map and the color score of each frame are adopted to estimate the reliability of tracking results. Third, we introduce the Siamese tracker to redetect targets in case of tracking failures. Finally, a target pyramid scheme is built to deal with scale variation. Extensive experiments on the OTB-2013, OTB-2015, TemplateColor, UAV123@10 fps, and VOT-2015 demonstrate that our proposed method outperforms favorably against the state-of-the-art methods with real-time tracking speed.

Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking

With efficient appearance learning models, discriminative correlation filter (DCF) has been proven to be very successful in recent video object tracking benchmarks and competitions. However, the existing DCF paradigm suffers from two major issues, i.e., spatial boundary effect and temporal filter degradation. To mitigate these challenges, we propose a new DCF-based tracking method. The key innovations of the proposed method include adaptive spatial feature selection and temporal consistent constraints, with which the new tracker enables joint spatial-temporal filter learning in a lower dimensional discriminative manifold. More specifically, we apply structured spatial sparsity constraints to multi-channel filters. Consequently, the process of learning spatial filters can be approximated by the lasso regularization. To encourage temporal consistency, the filter model is restricted to lie around its historical value and updated locally to preserve the global structure in the manifold. Last, a unified optimization framework is proposed to jointly select temporal consistency preserving spatial features and learn discriminative filters with the augmented Lagrangian method. Qualitative and quantitative evaluations have been conducted on a number of well-known benchmarking datasets such as OTB2013, OTB50, OTB100, Temple-Colour, UAV123, and VOT2018. The experimental results demonstrate the superiority of the proposed method over the state-of-the-art approaches.

SCRM: self-correlated representation model for visual tracking

Sparse representation (SR) as a seminal model for visual tracking explores the relationship between all candidates and the observed templates. Different from SR-based trackers, we propose a self-correlated representation model for robust visual tracking. Firstly, we learn a low-dimensional subspace representation from highly correlated templates to model the object, which aims at eliminating the redundant information and reducing the influence of noisy templates. Then, we represent the subspace by itself to learn the inner underlying features from subspace vectors. To further enhance model’s discriminating power, a new observation model is developed by considering both error distribution and large outliers. Experiments are conducted on some challenging video clips and demonstrate the favorable performance of our tracking system compared to some state-of-the-art representation-based trackers.

Visual Tracking via Auto-Encoder Pair Correlation Filter

Robust visual tracking is one of the most challenging problems in computer vision applications. However, the limited training data and the computational complexity have severely affected tracking performance. In this paper, we propose an auto-encoder pair model for visual tracking which is composed of source domain network and target domain network to help a more accurate localization. We adopt the dense circular samples of the object state to increase the number of training samples and prevent model overfitting. Meanwhile, a difference regularization term is also introduced into our framework to penalize the large appearance variations of the object in two domains. The alternating optimization is used to solve the optimization problems. Furthermore, our method alleviates the model update problem and improves the tracking speed by using long-term and short-term updating scheme. In addition, the target domain filter is updated by introducing the updated source domain filter to avoid the object drift. Comprehensive experiments on some challenging benchmarks demonstrate that our approach concurrently improves both tracking accuracy and speed.

A data-efficient deep learning approach for deployable multimodal social robots

The deep supervised and reinforcement learning paradigms (among others) have the potential to endow interactive multimodal social robots with the ability of acquiring skills autonomously. But it is still not very clear yet how they can be best deployed in real world applications. As a step in this direction, we propose a deep learning-based approach for efficiently training a humanoid robot to play multimodal games—and use the game of ‘Noughts and Crosses’ with two variants as a case study. Its minimum requirements for learning to perceive and interact are based on a few hundred example images, a few example multimodal dialogues and physical demonstrations of robot manipulation, and automatic simulations. In addition, we propose novel algorithms for robust visual game tracking and for competitive policy learning with high winning rates, which substantially outperform DQN-based baselines. While an automatic evaluation shows evidence that the proposed approach can be easily extended to new games with competitive robot behaviours, a human evaluation with 130 humans playing with the Pepper robot confirms that highly accurate visual perception is required for successful game play.

A Robust Visual Tracking Method for Unmanned Mobile Systems

This paper introduces a robust visual tracking of objects in complex environments with blocking obstacles and light reflection noises. This visual tracking method utilizes a transfer matrix to project image pixels back to real-world coordinates. During the image process, a color and shape test is used to recognize the object and a vector is used to represent the object, which contains the information of orientation and body length of the object. If the object is partially blocked by the obstacles or the reflection from the water surface, the vector predicts the position of the object. During the real-time tracking, a Kalman filter is used to optimize the result. To validate the method, the visual tracking algorithm was tested by tracking a submarine and a fish on the water surface of a water tank, above which three pieces of blur glass were blocking obstacles between the camera and the object. By using this method, the interference from the reflection of the side glass and the fluctuation of the water surface can be also avoided.

Robust visual tracking based on adaptive gradient descent optimization of a cost function with parametric models of appearance and geometry

In the field of visual tracking, there are many issues to consider which make the development of a robust tracking method very difficult, among these complications; the appearance change of the target, the fast motion, the background clutter, the camera motion, scale variation and the in plane Rotation. To override these problems, we develop an effective general framework for object tracking that addresses most of these issues. First the tracking problem is formulated in the form of a robust cost function which is a composition of the appearance and dynamic model, this formulation ensures the integration of the appearance and motion informations. Second the minimization is accomplished by the gradient descent optimization with adaptive step size prediction, the step size adaptation accelerates the optimization process and increases the accuracy. Throughout, we present experimental results made on different challenging sequences, the experimentations results demonstrate the efficiency and effectiveness of our methods.

Real-Time Event-Triggered Object Tracking in the Presence of Model Drift and Occlusion

In this paper, we propose a novel event-triggered tracking framework for fast and robust visual tracking in the presence of model drift and occlusion. The resulting tracker not only operates in real time, but also is resilient to tracking failures caused by factors such as fast motion and heavy occlusion. Specifically, the tracker consists of an event-triggered decision model as the core module that coordinates other functional modules, including a short-term tracker, occlusion and drift identification, target redetection, short-term tracker updating, and online discriminative learning for a detector. Each functional module is associated with a defined event that is triggered when the proposed conditions are met. The occlusion and drift identification module is intended to perform online evaluation of the short-term tracking. When a model drift event occurs, the target redetection module is activated by the event-triggered decision model to relocate the target and reinitialize the short-term tracker. The short-term tracker updating is carried out at each frame with a variable learning rate depending on the degree of occlusion. A sampling pool is constructed to store discriminative samples that are used to update the detector model. Extensive experiments on large benchmark datasets demonstrate that the proposed tracking algorithm can effectively detect model drift and restore tracking, and more importantly, it outperforms the state-of-the-art approaches in terms of accuracy, efficiency, and robustness.

Robust visual tracking via global context regularized Locality-constrained Linear Coding

Locality-constrained Linear Coding (LLC) based visual tracking can give a better and faster tracking performance than traditional sparse representation based tracking methods. However, the existing LLC based methods often use the anchor points near the target to build the sparse coding dictionary for local sparse coding. It may cause a problem that it is hard to discriminate the difference between the negative and positive anchor points in sparse coding dictionary when facing severe background clutter, illumination change and occlusion. In this paper, we propose a context aware sparse coding method to achieve robust visual tracking. The proposed method can prevent the negative anchor points from disturbing the classifier accuracy because it uses a global context regularizer to constrain the sparse coding value of those negative anchor points that are similar to the positive anchor points. Experiment results show that our method can achieve a better tracking performance than state-of-the-art tracking methods do.

Adaptive Weight Collaborative Complementary Learning for Robust Visual Tracking

Adaptive Weight Collaborative Complementary Learning for Robust Visual Tracking

Dual‐scale weighted structural local sparse appearance model for object tracking

It is a great challenge to develop an effective appearance model for robust visual tracking due to various interfering factors, such as pose change, occlusion, background clutter etc. More and more visual tracking methods tend to exploit the local appearance model to deal with the above challenges. In this study, the authors present a simple yet effective weighted structural local sparse appearance model, which can better describe the target appearance information through patch‐based generative weight. To further improve the robustness of tracking, they implement this appearance model on two‐scale patches. The two derived appearance models are then combined to form a collaborative model to play their advantages. Extensive experiments on the tracking benchmark dataset show that the proposed method performs favourably against several state‐of‐the‐art methods.

High-Performance Visual Tracking With Extreme Learning Machine Framework.

In real-time applications, a fast and robust visual tracker should generally have the following important properties: 1) feature representation of an object that is not only efficient but also has a good discriminative capability and 2) appearance modeling which can quickly adapt to the variations of foreground and backgrounds. However, most of the existing tracking algorithms cannot achieve satisfactory performance in both of the two aspects. To address this issue, in this paper, we advocate a novel and efficient visual tracker by exploiting the excellent feature learning and classification capabilities of an emerging learning technique, that is, extreme learning machine (ELM). The contributions of the proposed work are as follows: 1) motivated by the simplicity and learning ability of the ELM autoencoder (ELM-AE), an ELM-AE-based feature extraction model is presented, and this model can provide a compact and discriminative representation of the inputs efficiently and 2) due to the fast learning speed of an ELM classifier, an ELM-based appearance model is developed for feature classification, and is able to rapidly distinguish the object of interest from its surroundings. In addition, in order to cope with the visual changes of the target and its backgrounds, the online sequential ELM is used to incrementally update the appearance model. Plenty of experiments on challenging image sequences demonstrate the effectiveness and robustness of the proposed tracker.

Learning Local Structured Correlation Filters for Visual Tracking via Spatial Joint Regularization

Robust visual tracking is a fundamental problem in the field of computer vision and has a wide range of practical applications. Recent progress in developing robust tracking methods are mainly made upon discriminative correlation filters (DCF). However, most DCF-based methods develop their trackers under the assumption of a holistic appearance model, ignoring the underlying spatial local structural information. In this paper, we introduce the tree-structured group sparsity regularization into the DCF-based formula. The correlation filter to be learned is divided into hierarchical local groups. The relationship between the response and the circularly shifted target appearance is regularized by applying the l 1 -norm across the l 2 norm of the hierarchical local filter groups. Moreover, a local response consistency term is incorporated together with the structured sparsity to make each local filter group contributes equally to the final response. The accelerated proximal gradient method is employed to optimize this non-smooth composite regularization problem. Benefiting from the properties of circulant matrices, several key steps in the optimization process can be efficiently solved in the frequency domain. The experiments are conducted on four publicly available visual tracking benchmarks. Both quantitative and qualitative evaluations demonstrate that the proposed tracking method performs favorably against a number of state-of-the-art tracking methods.

Robust Visual Tracking With Spatial Regularization Kernelized Correlation Filter Constrained by a Learning Spatial Reliability Map

Robust Visual Tracking With Spatial Regularization Kernelized Correlation Filter Constrained by a Learning Spatial Reliability Map

Classifier Adaptive Fusion: Deep Learning for Robust Outdoor Vehicle Visual Tracking

Deep auto-encoder (DAE) models have been successfully used in object tracking due to its strong capability of feature representation. However, single deep auto-encoder model would not be robust enough to represent the appearance model of outdoor vehicle for its harsh working environment, such as illumination variation, occlusion, cluttered background and so on. In this paper, a novel multiple-DAE-based tracking approach, that is, classifier adaptive fusion for robust outdoor vehicle visual tracking approach is proposed under particle filter framework. Firstly, two deep auto-encoders are offline trained by gray-scale image and gradient image of the raw training images, respectively to obtain the stronger feature representation of gray-scale image and gradient image. Secondly, two classifiers are constructed using the encoder of the two well-trained deep auto-encoders and the output of the each classifier is used to compute the confidence of the corresponding particles. Finally, the confidence output of the two classifiers is fused and applied in online tracking, where, the fusion weight of the each classifier is computed according to the distribution of particles represented by different classifier. Extensive tracking experiments conducted on visual tracking benchmark (VTB) show that the proposed tracking algorithm outperforms 9 popular tracking algorithms in the challenge scenes of outdoor vehicle tracking such as illumination variation, occlusion, cluttered background and scale variation.