Visual Tracking Problem Research Articles

A Natural Feature Representation for Unstructured Environments

This paper addresses the long-standing problem of feature representation in the natural world for autonomous navigation systems. The proposed representation combines Isomap, which is a nonlinear manifold learning algorithm, with expectation maximization, which is a statistical learning scheme. The representation is computed off-line and results in a compact, nonlinear, non-Gaussian sensor likelihood model. This model can be easily integrated into estimation algorithms for navigation and tracking. The compactness of the model makes it especially attractive for deployment in decentralized sensor networks. Real sensory data from unstructured terrestrial and underwater environments are used to demonstrate the versatility of the computed likelihood model. The experimental results show that this approach can provide consistent models of natural environments to facilitate complex visual tracking and data-association problems.

Representation spaces in a visual-based human action recognition system

Visual tracking consists of locating or determining the configuration of a known object at each frame of a video sequence. Usually, the description of the whole scene involves the participation of multiple targets, their movements and interactions, etc., and the scenario particular features. This paper presents a visual tracking system framework oriented to provide a “near natural language” description of the involved targets in the scene actions. Our prototype focuses on the detection, tracking and feature extraction of a dynamic number of targets in a scenario along time. The design of any visual tracking system usually needs the injection of human knowledge at each transformed level of description, in order to produce from raw videos a linguistic scene summary. The main aim of this work was to make explicit the knowledge injection needed to link the low-level representations (associated to signals) to the high-level semantics (related to knowledge) in the visual tracking problem. As a result, the emerging semantic necessary at the two transformation level is analysed and presented. We have concentrated on the representation spaces for the memetic algorithm particle filter applied to multiple object tracking in annotated scenarios, oriented to video-based surveillance applications. Finally, some example applications on different surveillance scenarios are presented and discussed.

Vision-based tracking using intelligent excitation

This paper addresses the problem of visual tracking using a conventional model reference adaptive control framework for it. The adaptive controller is capable of maintaining a relative position for a trailer vehicle with respect to a lead vehicle of unknown size. The only information about the lead vehicle can be deduced from the subtended angle in the image plane of the trailing vehicle. Neither the dimension of the lead vehicle, nor its velocity are assumed to be known to the follower. The control objective of maintaining a prespecified relative range is achieved via an intelligent excitation signal, the properties of which are analysed in this paper. Simulations illustrate the theoretical results.

Bittracker—A Bitmap Tracker for Visual Tracking under Very General Conditions

This paper addresses the problem of visual tracking under very general conditions: a possibly non-rigid target whose appearance may drastically change over time; general camera motion; a 3D scene; and no a priori information except initialization. This is in contrast to the vast majority of trackers which rely on some limited model in which, for example, the target's appearance is known a priori or restricted, the scene is planar, or a pan tilt zoom camera is used. Their goal is to achieve speed and robustness, but their limited context may cause them to fail in the more general case. The proposed tracker works by approximating, in each frame, a PDF (probability distribution function) of the target's bitmap and then estimating the maximum a posteriori bitmap. The PDF is marginalized over all possible motions per pixel, thus avoiding the stage in which optical flow is determined. This is an advantage over other general-context trackers that do not use the motion cue at all or rely on the error-prone calculation of optical flow. Using a Gibbs distribution with respect to the first-order neighborhood system yields a bitmap PDF whose maximization may be transformed into that of a quadratic pseudo-Boolean function, the maximum of which is approximated via a reduction to a maximum-flow problem. Many experiments were conducted to demonstrate that the tracker is able to track under the aforementioned general context.

Visual Tracking in High-Dimensional State Space by Appearance-Guided Particle Filtering

In this paper, we propose a new approach, appearance-guided particle filtering (AGPF), for high degree-of-freedom visual tracking from an image sequence. This method adopts some known attractors in the state space and integrates both appearance and motion-transition information for visual tracking. A probability propagation model based on these two types of information is derived from a Bayesian formulation, and a particle filtering framework is developed to realize it. Experimental results demonstrate that the proposed method is effective for high degree-of-freedom visual tracking problems, such as articulated hand tracking and lip-contour tracking.

Multi-dimensional visual tracking using scatter search particle filter

Multi-dimensional visual tracking (MVT) problems include visual tracking tasks where the system state is defined by a high number of variables corresponding to multiple model components and/or multiple targets. A MVT problem can be modeled as a dynamic optimization problem. In this context, we propose an algorithm which hybridizes particle filters (PF) and the scatter search (SS) metaheuristic, called scatter search particle filter (SSPF), where the optimization strategies from SS are embedded into the PF framework. Scatter search is a population-based metaheuristic successfully applied to several complex combinatorial optimization problems. The most representative optimization strategies from SS are both solution combination and solution improvement. Combination stage enables the solutions to share information about the problem to produce better solutions. Improvement stage makes also possible to obtain better solutions by exploring the neighborhood of a given solution. In this paper, we have described and evaluated the performance of the scatter search particle filter (SSPF) in MVT problems. Specifically, we have compared the performance of several state-of-the-art PF-based algorithms with SSPF algorithm in different instances of 2D articulated object tracking problem and 2D multiple object tracking. Some of these instances are from the CVBase’06 standard database. Experimental results show an important performance gain and better tracking accuracy in favour of our approach.

Visual Tracking of a Maneuvering Target

This paper presents an acceleration-command generation and its implementation for an aerial vehicle in the problem of visual target tracking, when the target is free to make any maneuver with arbitrary but otherwise bounded acceleration. The acceleration command is generated by an adaptive disturbance rejection control algorithm that uses only visual information about the target, obtained by a monocular camera mounted on the aerial vehicle. This information consists of the maximal image size in pixels and the pixel coordinates of the image centroid in the image plane. The acceleration command is translated into forward velocity and attitude angle commands, which are afterward used to obtain actual control surface deflections for the aerial vehicle. Simulations of a full nonlinear model verify the theoretical statements.

An EM/E-MRF algorithm for adaptive model based tracking in extremely poor visibility

This paper addresses the problems of visual tracking in conditions of extremely poor visibility. The human visual system can often correctly interpret images that are of such poor quality that they contain insufficient explicit information to do so. We assert that such systems must therefore make use of prior knowledge in several forms. A tracking algorithm is presented which combines observed data (the current image) with predicted data derived from prior knowledge of the object being viewed and an estimate of the camera’s motion. During image segmentation, a predicted image is used to estimate class conditional distribution models and an Extended-Markov Random Field technique is used to combine observed image data with expectations of that data within a probabilistic framework. Interpretations of scene content and camera position are then mutually improved using Expectation Maximisation. Models of background and tracked object are continually relearned and adapt iteratively with each new image frame. The algorithm is tested using real video sequences, filmed in poor visibility conditions with complete pre-measured ground-truth data.

ROBUST CONTROLLER DESIGN FOR IMAGE-BASED VISUAL TRACKING SYSTEM

This paper focuses on visual servoing systems, particularly on image-based visual tracking problem. Basically, in image-based visual tracking, the error signal is computed in the image plane and the command signals are generated with respect to such error by means of a visual Jacobian. The primary controller design challenge of image-based visual tracking system is the robustness with respect to calibration parameter uncertainties. In this paper, a new nonlinear controller design technique is proposed for stable image-based visual tracking. The new controller design takes advantage of the sliding mode control and it yields more robust on calibration parameter uncertainties. The range of admissible uncertainties is enlarged to modify the uncertainty bound matrix. The proposed design technique was applied on a real system and effectiveness of the method was validated through experiments.

Multi-modal tracking using texture changes

We present a method for efficiently generating a representation of a multi-modal posterior probability distribution. The technique combines ideas from RANSAC and particle filtering such that the 3D visual tracking problem can be partitioned into two levels, while maintaining multiple hypotheses throughout. A simple texture change-point detector finds multiple hypotheses for the position of image edgels. From these, multiple locations for each scene edge are generated. Finally, we determine the best pose of the whole structure. While the multi-modal representation is strongly related to particle filtering techniques, this approach is driven by data from the image. Hence the resulting system is able to perform robust visual tracking of all six degrees of freedom in real time. Real video sequences are used to compare the complete tracking system to previous systems.

Spatio-Temporal Graphical-Model-Based Multiple Facial Feature Tracking

It is challenging to track multiple facial features simultaneously when rich expressions are presented on a face. We propose a two-step solution. In the first step, several independent condensation-style particle filters are utilized to track each facial feature in the temporal domain. Particle filters are very effective for visual tracking problems; however multiple independent trackers ignore the spatial constraints and the natural relationships among facial features. In the second step, we use Bayesian inference—belief propagation—to infer each facial feature's contour in the spatial domain, in which we learn the relationships among contours of facial features beforehand with the help of a large facial expression database. The experimental results show that our algorithm can robustly track multiple facial features simultaneously, while there are large interframe motions with expression changes.

Navigating inner space: 3-D assistance for minimally invasive surgery

Since its inception about three decades ago, modern minimally invasive surgery has made huge advances in both technique and technology. However, the minimally invasive surgeon is still faced with daunting challenges in terms of visualization and hand-eye coordination. At the Center for Computer Integrated Surgical Systems and Technology (CISST) we have been developing a set of techniques for assisting surgeons in navigating and manipulating the three-dimensional space within the human body. In order to develop such systems, a variety of challenging visual tracking, reconstruction and registration problems must be solved. In addition, this information must be tied to methods for assistance that improve surgical accuracy and reliability but allow the surgeon to retain ultimate control of the procedure and do not prolong time in the operating room. In this article, we present two problem areas, eye microsurgery and thoracic minimally invasive surgery, where computational vision can play a role. We then describe methods we have developed to process video images for relevant geometric information, and related control algorithms for providing interactive assistance. Finally, we present results from implemented systems.

Dynamic Color Tracking Based on Probabilistic Data Association

This paper presents a color tracking method based on probabilistic data association in order to resolve difficult and complicated visual tracking problem, such as a changing of target’s representation, a clutter of environments and an interaction of target and camera. Because the probabilistic data association is flexible and suitable for ambiguous and missing data, which generates the difficulties of visual tracking, some methods of probabilistic data association could be combined and applied in this tracking method to find the solutions of these difficulties. Due to using sequential Monte Carlo framework, this tracking method is applied to tracking of changeful target by handling the related information between every frame in image sequences. In order to improve tracking accuracy, this method utilizes factorized sampling algorithm to express target characters as sample-set. Moreover, this method benefits from HSV color model and captures the color natures of object like human to enhance the color-sensing capability of computer. Hence, this method could be considered as self-learning system and imitate the based human vision function - tracking. The tracking system applying this method is implemented in real-time at around 15Hz with 640 × 480 pixels image. The results show that the self-learning and real-time system is able to track a target robustly with enough accuracy and automatically control the camera’s pan, tilt and zoom to remain the object centered in the field of vision.

Visual Tracking and Object Recognition

In this paper, we will discuss the closely related problems of visual tracking and object recognition. The solution involves adaptive and robust control in conjunction with multiscale methods from signal processing, and shape recognition theory from computer vision. We should note that the problem of visual tracking differs from standard tracking problems in that the feedback signal is measured using imaging sensors. In particular, it has to be extracted via computer vision algorithms and interpreted by a reasoning algorithm before being used in the control loop. Recognition will be accomplished using a novel invariant theoretic approach to shape analysis.

アフィン変換テンプレート群の動的遷移に基づくビジュアルトラッキング

Visual tracking plays an important role in various robotic tasks such as monitoring an object in the manipulation or tracking a target in navigation of a mobile robot. This paper presents a view-based visual tracking system. The view-based systems are more general than the other approaches which require analysis of 2D image features and the matching with the 3D models. For the image correlation, we use a visual tracking hardware based on a template matching algorithm. One problem of the view-based visual tracking is how to cope with the changes of the template's appearance in the 3D environment. Consequently, the system generates affine transformed templates corresponding to various orientation of the target image plane. We use geodesic domes not only for an uniform sampling of the 3D orientations of the template image plane, but also for dynamic transition in groups of the candidate templates to utilize the latest matching results. Experimental results demonstrate the usefulness of the proposed system.

A Neural Controller based Binocular Vision System for Object Tracking

In this paper, we present a learning control approach to the problem of visual tracking using an active binocular vision system. The simulated vision system used here is composed of two cameras which have freedom of pan and are mounted on a plate that can tilt. A self organizing map is trained to learn the mapping between visual feedback as received from the simulated CCD cameras and the incremental values in the orientation angles of the vision system to fixate continuously on a mobile target. A direct implication of this is that the object should be centered at the vergence point of the cameras. This is accomplished by ensuring minimum disparity of the target vector from the center of the images of both the cameras. The learning algorithm is presented and the simulation study shows an average focussing error of 0.15 pixels after 1200 iterations for a small, defined joint angle space of the vision system. Thus, a significant contribution of this work is the application of a self organizing map in dynamic object tracking.

The framework of controlled active vision

One of the most desirable characteristics of a robotic manipulator is its flexibility. Flexibility and adaptability can be achieved by incorporating vision and generally, sensory information in the feedback loop. Our research introduces a framework called controlled active vision for efficient integration of the vision sensor in the feedback loop. This framework was applied to the problem of robotic visual tracking and servoing, and the results were very promising. Full 3-D robotic visual tracking was achieved at rates of 30 Hz. Most importantly, the tracking was successful even under the assumption of poor calibration of the eye-in-hand system. This paper extends this framework to other problems of sensor-based robotics, such as the derivation of depth maps from controlled motion; the vision-assisted grasping; the active calibration of the system robot-camera; and the computation of the relative pose of the target with respect to the camera. We address these problems by combining adaptive control techniques with computer vision algorithms. The paper concludes with a discussion on several relative issues such as the stability and robustness of the proposed algorithms and the problem of incorporating stereo information in the existing algorithms in order to increase the accuracy of the estimated depth.

Experimental Verification of H∞ Filter for Robust Visual Tracking

This paper is concerned with the performance verification of the H∞ filter for a visual tracking problem. The efficacy of the H∞ filter is validated experimentally with a hand-eye coordinated robot manipulator system using stereo. After the introduction of an experimental architecture, a mathematical model for the visual tracking problem is described. Then, the estimation and control method is discussed based on the H∞ filter algorithm. Finally, an empirical comparison is made with the H∞ filter and the Kalman filter for the estimation problem arising in the visual tracking. The experimental results reveal its potential efficacy of the H∞ filter.

TRICLOPS: A tool for studying active vision

The design, performance, and application of The Real-time, Intelligently ControLled, Optical Positioning System (TRICLOPS) are described in this article. TRICLOPS is a multiresolution trinocular camera-pointing system which provides a center wide-angle view camera and two higher-resolution vergence cameras. It is a direct-drive system that exhibits dynamic performance comparable to the human visual system. The mechanical design and performance of various active vision systems are discussed and compared to those of TRICLOPS. The multiprocessor control system for TRICLOPS is described. The kinematics of the device are also discussed and calibration methods are given. Finally, as an example of real-time visual control, a problem in visual tracking with TRICLOPS is examined. In this example, TRICLOPS is shown to be capable of tracking a ball moving at 3 m/s, which results in rotational velocities of the vergence cameras in excess of 6 rad/s (344 deg/s).