Visual Tracking Framework Research Articles

Deformable Patch-based NCC Measure for Visual Tracking

This paper tries to solve the visual tracking problem in a simple and intuitive way. In particular, we extend the traditional NCC measure with deformable part-based models (DPM) to get a new image similarity measurement and feed it into the particle filtering framework for visual object tracking. The proposed target model consists of a root template and a set of patch templates. The root is the whole target image, and patches are sub-regions of the target image. NCC responses are calculated for the root template with test images, and every patch template with their corresponding sub-regions of test images. The likelihood between the target and a test image is calculated by integrating all template responses. Instead of directly accumulating all responses, we employ the idea of DPM that combines global and local responses using an elastic model to get a robust image likelihood measure. We also explore an exhaustive patch representation mechanism by introducing the dense NCC feature map. The proposed feature map enables the usage of all patches in the target region for target description and similarity measure. Finally, we provide both qualitative and quantitative experiments against state-of-the-arts on the TB50 dataset. Experimental results show that the proposed similarity measure is effective for visual tracking under challenging scenarios.

A novel biologically inspired computational framework for visual tracking task

Abstract Visual tracking is a process in which the location of one or several objects is estimated in a video sequence based on their appearance. High diversity of states and conditions of a moving stimulus and the existence of abundant challenges such as a cluttered background, variations in the objects appearance, and occlusion make this problem very complicated. Although a wide range of diverse algorithms have been introduced in this field, even the state-of-art ones have not been able to achieve high accuracy in facing different challenges of this problem. Nevertheless, by observing the biological world and paying attention to the living beings, particularly humans, it is revealed that the biological visual system performs the visual tracking task ideally. Therefore, inspiring from this biological system and especially its spatiotemporal processing, motion perception, attention, and saliency mechanism, in this paper a new biologically inspired visual tracking framework is proposed. The suggested framework includes a six-phase process in which the executive blocks, their connections, and information flow among them are inspired by the visual cortex. Using visual features extracted in primary layers of the visual cortex as well as the local-global and sparse representation of the object of interest make possible to employ the proposed framework in different conditions. In this paper, the theory of the suggested framework and its biological background is explained. Investigating the procedure of proposed framework from the viewpoint of employed features, spatiotemporal process, and the quiddity of local-global processing indicated that this framework enjoys the ability to manage most of the mentioned challenges, and can robustly track the object of interest in real conditions and application. This paper seeks to bridge biological and machine visions in order to bring the robustness, speed, and accuracy of biological vision into the artificial intelligence world.

Robust visual tracking via constrained correlation filter coding

Unconstrained correlation filters based trackers achieve superior performance with high speed in visual tracking. However, such unconstrained correlation filters do not impose any hard constraint to their responses to have a certain value, which brings about classification ambiguity on intractable samples (i.e., two similar samples from different classes). To tackle this issue, in this paper, constrained correlation filter is introduced into visual tracking framework to alleviate classification ambiguity for more accurate target location. By imposing distinguishable hard constraints on the response map to different classes, a supervised coding method is proposed to encode various candidate samples by a discriminative filter bank. The learned high-level feature vectors are sent to a Naive Bayes classifier to separate the target from the background. Besides, parameters updating schemes in the constrained filter and classifier are introduced to adapt to appearance changes of the target with less possibility of drifting. Both qualitative and quantitative evaluations on Object Tracking Benchmark (OTB) show that the proposed tracking method achieves favorable performance compared with other state-of-the-art methods.

Robust visual tracking framework in the presence of blurring by arbitrating appearance- and feature-based detection

Abstract This paper proposes a new visual tracking framework and demonstrates its merits via mobile robot experiments. An image sequence from the vision system of a mobile robot is not static when a mobile robot is moving, since slipping and vibration occur. These problems cause image blurring. Therefore, in this paper, we address the problem of robust object tracking under blurring and introduce a novel robust visual tracking framework based on the arbitration of the AdaBoost-based detection method and the appearance-based detection method to overcome the blurring problem. The proposed framework consists of three parts: (1) distortion error compensation and feature extraction using the Modified Discrete Gaussian–Hermite Moment (MDGHM) and fuzzy-based distortion error compensation, (2) object detection using arbitration of appearance- and feature-based object detection, and (3) object tracking using a Finite Impulse Response (FIR) filter. To demonstrate the performance of the proposed framework, mobile robot visual tracking experiments are carried out. The results show that the proposed framework is more robust against blurring than the conventional feature- and appearance-based methods.

Robust visual tracking based on generative and discriminative model collaboration

Effective object appearance model is one of the key issues for the success of visual tracking. Since the appearance of a target and the environment changes dynamically, the majority of existed visual tracking algorithms tend to drift away from targets. To address this issue, we propose a robust tracking algorithm by integrating the generative and discriminative model. The object appearance model is made up of generative target model and a discriminative classifier. For the generative target model, we adopt the weighted structural local sparse appearance model combining patch based gray value and Histogram of Oriented Gradients feature as the patch dictionary. By sampling positives and negatives, alignment-pooling features are obtained based on the patch dictionary through local sparse coding, then we use support vector machine to train the discriminative classifier. The proposed method is embedded into a Bayesian inference framework for visual tracking. A combined matching method is adopted to improve the proposal distribution of the particle filter. Moreover, in order to adapt the situation change, the patch dictionary and discriminative classifier are updated by incremental learning every five frames. Experimental results on some publicly available benchmarks of video sequences demonstrate the accuracy and effectiveness of our tracker.

Dual Group Structured Tracking

The sparse representation (SR)-based tracking framework generally considers the testing candidates and dictionary atoms individually, thus failing to model the structured information within data. In this paper, we present a robust tracking framework by exploiting the dual group structure of both candidate samples and dictionary templates, and formulate the SR at group level. The similar samples are encoded simultaneously by a few atom groups, which induces the inter-group sparsity, and also each group enjoys different internal sparsity. In this way, not only the potential commonality shared by the related candidates is taken into account but also the individual differences between samples are reflected. Then, we provide two effective optimization methods to solve our formulation by block-coordinate gradient descent and alternating direction method of multipliers, respectively, and make a comparison between them in terms of both effectiveness and efficiency. Finally, we embed the dual group structure model into the particle filter framework for visual tracking. Extensive experimental results demonstrate that our tracker achieves favorable performance against the state-of-the-art tracking methods.

Integrating Local Binary Patterns into Normalized Moment of Inertia for Updating Tracking Templates

This paper presents an efficient visual tracking framework which is robust to rotation, scale variation and occlusion. The target template is characterized by Local binary patterns (LBP), which exhibit invariance to rotation. The LBP features are then integrated into the Normalized moment of inertia (NMI) to decide whether the template requires update. This procedure enables an adaptive template matching strategy which addresses the tracking failures arising from scale variations. Kalman filtering is exploited for predicting the trajectory of the target when it is occluded. The matching efficiency is achieved by applying a locally pyramid searching scheme. Experimental results validate the efficiency and effectiveness of our tracking framework.

Effective real-time vehicle tracking using discriminative sparse coding on local patches

A visual tracking framework that provides an object detector and tracker, which focuses on effective and efficient visual tracking in surveillance of real-world intelligent transport system applications, is proposed. The framework casts the tracking task as problems of object detection, feature representation, and classification, which is different from appearance model-matching approaches. Through a feature representation of discriminative sparse coding on local patches called DSCLP, which trains a dictionary on local clustered patches sampled from both positive and negative datasets, the discriminative power and robustness has been improved remarkably, which makes our method more robust to a complex realistic setting with all kinds of degraded image quality. Moreover, by catching objects through one-time background subtraction, along with offline dictionary training, computation time is dramatically reduced, which enables our framework to achieve real-time tracking performance even in a high-definition sequence with heavy traffic. Experiment results show that our work outperforms some state-of-the-art methods in terms of speed, accuracy, and robustness and exhibits increased robustness in a complex real-world scenario with degraded image quality caused by vehicle occlusion, image blur of rain or fog, and change in viewpoint or scale.

Robust object tracking by online Fisher discrimination boosting feature selection

Large appearance changes in visual tracking affect the tracking performance severely. To address this challenge, in this paper we develop an effective appearance model with the highly discriminative features. We propose an online Fisher discrimination boosting feature selection mechanism, which selects features that reduce the with-in scatter while enlarging the between-class scatter, thereby enhancing the discriminative capability between the target and background. Moreover, we utilize a particle filtering framework for visual tracking, in which the weights of candidate particles take into account the context information around the particles, thereby enhancing the robustness of tracking. In order to increase efficiency, a coarse-to-fine search strategy is exploited to efficiently and accurately locate the target. Extensive experiments on the CVPR2013 tracking benchmark demonstrate the competitive performance of our algorithm over other representative algorithms in terms of accuracy and robustness.

Struck: Structured Output Tracking with Kernels.

Adaptive tracking-by-detection methods are widely used in computer vision for tracking arbitrary objects. Current approaches treat the tracking problem as a classification task and use online learning techniques to update the object model. However, for these updates to happen one needs to convert the estimated object position into a set of labelled training examples, and it is not clear how best to perform this intermediate step. Furthermore, the objective for the classifier (label prediction) is not explicitly coupled to the objective for the tracker (estimation of object position). In this paper, we present a framework for adaptive visual object tracking based on structured output prediction. By explicitly allowing the output space to express the needs of the tracker, we avoid the need for an intermediate classification step. Our method uses a kernelised structured output support vector machine (SVM), which is learned online to provide adaptive tracking. To allow our tracker to run at high frame rates, we (a) introduce a budgeting mechanism that prevents the unbounded growth in the number of support vectors that would otherwise occur during tracking, and (b) show how to implement tracking on the GPU. Experimentally, we show that our algorithm is able to outperform state-of-the-art trackers on various benchmark videos. Additionally, we show that we can easily incorporate additional features and kernels into our framework, which results in increased tracking performance.

Visual tracking for the recovery of multiple interacting plant root systems from X-ray $$\upmu $$ μ CT images

We propose a visual object tracking framework for the extraction of multiple interacting plant root systems from three-dimensional X-ray micro computed tomography images of plants grown in soil. Our method is based on a level set framework guided by a greyscale intensity distribution model to identify object boundaries in image cross-sections. Root objects are followed through the data volume, while updating the tracker's appearance models to adapt to changing intensity values. In the presence of multiple root systems, multiple trackers can be used, but need to distinguish target objects from one another in order to correctly associate roots with their originating plants. Since root objects are expected to exhibit similar greyscale intensity distributions, shape information is used to constrain the evolving level set interfaces in order to lock trackers to their correct targets. The proposed method is tested on root systems of wheat plants grown in soil.

Visual Tracking Based on the Adaptive Color Attention Tuned Sparse Generative Object Model

This paper presents a new visual tracking framework based on an adaptive color attention tuned local sparse model. The histograms of sparse coefficients of all patches in an object are pooled together according to their spatial distribution. A particle filter methodology is used as the location model to predict candidates for object verification during tracking. Since color is an important visual clue to distinguish objects from background, we calculate the color similarity between objects in the previous frames and the candidates in current frame, which is adopted as color attention to tune the local sparse representation-based appearance similarity measurement between the object template and candidates. The color similarity can be calculated efficiently with hash coded color names, which helps the tracker find more reliable objects during tracking. We use a flexible local sparse coding of the object to evaluate the degeneration degree of the appearance model, based on which we build a model updating mechanism to alleviate drifting caused by temporal varying multi-factors. Experiments on 76 challenging benchmark color sequences and the evaluation under the object tracking benchmark protocol demonstrate the superiority of the proposed tracker over the state-of-the-art methods in accuracy.

Tracker-Level Fusion for Robust Bayesian Visual Tracking

We propose a tracker-level fusion framework for robust visual tracking. The framework combines trackers addressing different tracking challenges to improve the overall performance. A novelty of the proposed framework is the inclusion of an online performance measure to identify the track quality level of each tracker so as to guide the fusion. The fusion is then based on appropriately mixing the prior state of the trackers. Moreover, the track-quality level is used to update the target appearance model. We demonstrate the framework with two Bayesian trackers on video sequences with various challenges and show its robustness compared with the independent use of the two individual trackers, and also compared with state-of-the-art trackers that use tracker-level fusion.

Visual tracking with multifeature joint sparse representation

We present a visual tracking method with feature fusion via joint sparse presentation. The proposed method describes each target candidate by combining different features and joint sparse representation for robustness in coefficient estimation. Then, we build a probabilistic observation model based on the approximation error between the recovered candidate image and the observed sample. Finally, this observation model is integrated with a stochastic affine motion model to form a particle filter framework for visual tracking. Furthermore, a dynamic and robust template update strategy is applied to adapt the appearance variations of the target and reduce the possibility of drifting. Quantitative evaluations on challenging benchmark video sequences demonstrate that the proposed method is effective and can perform favorably compared to several state-of-the-art methods.

Active Contour-Based Visual Tracking by Integrating Colors, Shapes, and Motions Using Level Sets

Using a camera,the visual object tracking is one of the most important process in searching the spot of moving object over the time. In the case of the object moves fast relative to the frame rate,the visual object tracking is difficult task. The active contour evolution algorithm which is used for the tracking of object in a given frame of an image sequence. Active contour based visual object tracking using the level sets is proposed which does not consider the camera either stationary or moving. We present a framework for active contour-based visual object tracking using the level sets. The main components of our framework consist of the contour-based tracking initialization, colour-based contour evolution, the adaptive shape-based contour evolution for the non-periodic motions, the dynamic shape-based contour evolution for the periodic motions and handling of the abrupt motions. For the contour-based tracking initialization, we use an optical flow-based algorithm for the automatically initializing contours at the first frame. In the color-based contour evolution, we use Markov random field theory to measure correlations between values of the neighboring pixels for the posterior probability estimation.In the adaptive shape-based contour evolution, we combined the global shape information and the local color information to hierarchically develop gradually the contour, and a flexible shape updating model is made. In the dynamic shape based contour evolution, a shape mode transition matrix is gain to characterize the temporal correlations of the object shapes. In the handling of abrupt motions, particle swarm optimization (PSO) is used to capture the global motion which is applied to the contour in the current frame to produce an initial contour in the next frame.

Visual Tracking for Abrupt Motions of Human Sperm Using Smoothing Stochastic Approximate Monte Carlo

Characteristic of sperm movement or sperm motility is one of important quality of sperm. Computer-aided Sperm Analysis (CASA) systems have attempted to give more accurate information about sperm motility. For collecting a single good sperm in Intracytoplasmic Sperm Injection (ICSI) procedures, achieving correct sperm trajectory is necessary. On the other hand, visual tracking of a sperm is a challenging issue because: (i) sperms have similar size and shape, (ii) a good sperm moves fast and has unpredictable motions. Furthermore, our sperm videos were taken by regular cameras which have low frame rate (about 20 fps), which cause the sperm-motion more abrupt. To address this problem, we propose searching driven stochastic sampling framework for visual tracking of abrupt motions. In here, searching algorithm will give promising regions of the target, and thus can replace the role of transition model in tracking system. For searching, we employ Coherency Sensitive Hashing (CSH) in a window to constrain search space in each frame, called as Search Window. And then for stochastic sampling framework, we adopt Smoothing Stochastic Approximate Monte Carlo (SSAMC) to specifically finding the target. The experimental results on human-sperm sequences show that: (i) comparing to our previous tracker which use geometric transition dynamic model, the proposed tracker can handle abrupt motions robustly, (ii) by using appropriate Search Window, the tracker can localize the sperm target and restrict observation of other similar sperms.

Visual Tracking Using an Insect Vision Embedded Particle Filter

Particle filtering (PF) based object tracking algorithms have drawn great attention from lots of scholars. The core of PF is to predict the possible location of the target via the state transition model. One commonly adopted approach is resorting to prior motion cues under the smooth motion assumption, which performs well when the target moves with a relatively stable velocity. However, it would possibly fail if the target is undergoing abrupt motion. To address this problem, inspired by insect vision, we propose a simple yet effective visual tracking framework based on PF. Utilizing the neuronal computational model of the insect vision, we estimate the motion of the target in a novel way so as to refine the position state of propagated particles using more accurate transition mode. Furthermore, we design a novel sample optimization framework where local and global search strategies are jointly used. In addition, we propose a new method to monitor long duration severe occlusion and we could recover the target. Experiments on publicly available benchmark video sequences demonstrate that the proposed tracking algorithm outperforms the state-of-the art methods in challenging scenarios, especially for tracking target which is undergoing abrupt motion or fast movement.

Survey of single‐target visual tracking methods based on online learning

Visual tracking is a popular and challenging topic in computer vision and robotics. Owing to changes in the appearance of the target and complicated variations that may occur in various scenes, online learning scheme is necessary for advanced visual tracking framework to adopt. This paper briefly introduces the challenges and applications of visual tracking and focuses on discussing the state‐of‐the‐art online‐learning‐based tracking methods by category. We provide detail descriptions of representative methods in each category, and examine their pros and cons. Moreover, several most representative algorithms are implemented to provide quantitative reference. At last, we outline several trends for future visual tracking research.

Adaptive NormalHedge for robust visual tracking

In this paper, we propose a novel visual tracking framework, based on a decision-theoretic online learning algorithm namely NormalHedge. To make NormalHedge more robust against noise, we propose an adaptive NormalHedge algorithm, which exploits the historic information of each expert to perform more accurate prediction than the standard NormalHedge. Technically, we use a set of weighted experts to predict the state of the target to be tracked over time. The weight of each expert is online learned by pushing the cumulative regret of the learner towards that of the expert. Our simulation experiments demonstrate the effectiveness of the proposed adaptive NormalHedge, compared to the standard NormalHedge method. Furthermore, the experimental results of several challenging video sequences show that the proposed tracking method outperforms several state-of-the-art methods.

Correlation, Kalman filter and adaptive fast mean shift based heuristic approach for robust visual tracking

Correlation tracker is computation intensive (if the search space or the template is large), has template drift problem, and may fail in case of fast maneuvering target, rapid changes in its appearance, occlusion suffered by it and clutter in the scene. Kalman filter can predict the target coordinates in the next frame, if the measurement vector is supplied to it by a correlation tracker. Thus, a relatively small search space can be determined where the probability of finding the target in the next frame is high. This way, the tracker can become fast and reject the clutter, which is outside the search space in the scene. However, if the tracker provides wrong measurement vector due to the clutter or the occlusion inside the search region, the efficacy of the filter is significantly deteriorated. Mean-shift tracker is fast and has shown good tracking results in the literature, but it may fail when the histograms of the target and the candidate region in the scene are similar (even when their appearance is different). In order to make the overall visual tracking framework robust to the mentioned problems, we propose to combine the three approaches heuristically, so that they may support each other for better tracking results. Furthermore, we present novel method for (1) appearance model updating which adapts the template according to rate of appearance change of target, (2) adaptive threshold for similarity measure which uses the variable threshold for each forthcoming image frame based on current frame peak similarity value, and (3) adaptive kernel size for fast mean-shift algorithm based on varying size of the target. Comparison with nine state-of-the-art tracking algorithms on eleven publically available standard dataset shows that the proposed algorithm outperforms the other algorithms in most of the cases.