Tracking In Image Sequences Research Articles

Automatic multiple moving humans detection and tracking in image sequences taken from a stationary thermal infrared camera

Several Particle Filter (PF)-based methods for human tracking in thermal IR image sequences have been proposed in the literature. Unfortunately, the majority of these methods are developed for tracking only a single human. Moreover, this human is manually pre-selected in the first frame of the image sequence, which is not practical for the real case of intelligent and efficient video surveillance system that needs tracking more than one human and without any external operator intervention. To contribute to addressing this need, in this paper, we propose a novel PF-based method that detects and tracks multiple moving humans using a thermal IR camera, without prior knowledge about their number and initial locations in the monitored scene. This method consists of three main parts. In the first one, all the moving objects are extracted from the image sequence by using the Gaussian Mixture Model (GMM) and then, for each extracted object, a combined shape, appearance, spatial and temporal-based similarity function that allows us to detect a human without any prior training of a mathematical model is calculated. The second part consists in tracking the human previously detected by using a PF and an adaptive combination of spatial, intensity, texture and motion velocity cues. In each cue, a model for the detected human is created, and when new observations arrive in the next frames, the similarity distances between each created model and the observed moving regions are calculated. The human tracking is achieved by combining individual similarity distances using adaptive weights, into a PF algorithm. The third part is devoted to detect and handle occlusions by using simple heuristic rules and grayscale Vertical Projection Histogram (VPH). Each part of the proposed method was separately tested on a set of real-world thermal IR image sequences containing background clutters, appearance and disappearance of multiple moving objects, occlusions, illumination and scales changes. A comparative study with several state-of-the-art methods has shown that the proposed method performs consistently better in terms of Center Location Error (CLE) and the Success Rate (SR), and it can also run at speed of about 15 Frame Per Second per human, which is considerably enough for real-time applications.

Trail optimization framework to detect nonlinear object motion in video sequences

Detecting and tracking multiple moving objects in a sequence of video images is an important application in automated surveillance systems, service robots, target discrimination, traffic monitoring, etc. Since these systems require real-time processing, providing an efficient method with lower computational complexity is a challenge. Besides the combination of rudimentary techniques which works well for linear objects, this paper presents an operative technique to recognize and track multiobject moving in a video sequence without using additional trackers. Instead of processing the entire video sequence, candidate keyframes are identified and the foreground moving objects are detected. To solve the nonlinear tracking problem, this paper presents an optimized twin algorithm—amended Kalman with Hungarian algorithm—to track multiple moving objects on their center of gravity in the minimal bounding box. Kalman filter predicts the location of the foreground objects in various orientation bins to extract the short-term movement of the foreground objects in possible trajectory paths. Hungarian algorithm locates the presence/absence of nonlinear objects in the tracks. Location prediction and tracking of moving objects only on the candidate frames reduces the computation time. Therefore, it is a good alternative method for nonlinear motion estimation that is likely required for multiobject identity tracking in image sequences. This approach achieves high accuracy and reduces additional computations comparable to state-of-the-art online trackers. The comparison also proved that the proposed method has better precision–recall values with computation simplicity.

An Improved Digital Image Correlation Calculation in the Case of Substantial Lighting Variation

In the last decade, digital image correlation (DIC) has become the most widely used non-contact technique for measuring full-field motion and deformation in image sequences. As permanent hardware improvements make it possible to readily obtain digital images with higher and higher resolutions at ever-increasing frame rates, the efficiency of image registration methods is becoming increasingly important. Another aspect of this problem is the desire to improve the accuracy of these methods, which in general implies additional computational time. This problem is addressed by applying the inverse compositional Gauss-Newton (IC-GN) algorithm, which is modified to include the parametric sum of squared differences (PSSD) correlation criterion. However, the classical IC-GN algorithm with the sum of squared differences (SSD) correlation criterion cannot be used in cases in which lighting variations are of considerable importance. In this paper, a solution to this disadvantage is presented. The lighting variation is directly included in the IC-GN minimization procedure, which reduces the additional calculation time. Moreover, this IC-GN algorithm does not need to re-evaluate and invert the Hessian matrix. This results in a more computationally efficient algorithm. Experimental results show that the proposed algorithm is insensitive to the linear intensity variation in the target image; consequently, it has a faster convergence rate than the IC-GN algorithm with the zero-mean normalized sum of squared differences (ZNSSD) correlation criterion. As a result, computational time can be reduced by up to 30% in the case of convergence condition 𝜖=0.001. Owing to the above mentioned advantages of the proposed IC-GN algorithm, it could become a new standard for efficient and robust full-field displacement tracking in image sequences with varying lighting intensity.

Probabilistic multi-person localisation and tracking in image sequences

The localisation and tracking of persons in image sequences in commonly guided by recursive filters. Especially in a multi-object tracking environment, where mutual occlusions are inherent, the predictive model is prone to drift away from the actual target position when not taking context into account. Further, if the image-based observations are imprecise, the trajectory is prone to be updated towards a wrong position. In this work we address both these problems by using a new predictive model on the basis of Gaussian Process Regression, and by using generic object detection, as well as instance-specific classification, for refined localisation. The predictive model takes into account the motion of every tracked pedestrian in the scene and the prediction is executed with respect to the velocities of neighbouring persons. In contrast to existing methods our approach uses a Dynamic Bayesian Network in which the state vector of a recursive Bayes filter, as well as the location of the tracked object in the image, are modelled as unknowns. This allows the detection to be corrected before it is incorporated into the recursive filter. Our method is evaluated on a publicly available benchmark dataset and outperforms related methods in terms of geometric precision and tracking accuracy.

A GAUSSIAN PROCESS BASED MULTI-PERSON INTERACTION MODEL

Abstract. Online multi-person tracking in image sequences is commonly guided by recursive filters, whose predictive models define the expected positions of future states. When a predictive model deviates too much from the true motion of a pedestrian, which is often the case in crowded scenes due to unpredicted accelerations, the data association is prone to fail. In this paper we propose a novel predictive model on the basis of Gaussian Process Regression. The model takes into account the motion of every tracked pedestrian in the scene and the prediction is executed with respect to the velocities of all interrelated persons. As shown by the experiments, the model is capable of yielding more plausible predictions even in the presence of mutual occlusions or missing measurements. The approach is evaluated on a publicly available benchmark and outperforms other state-of-the-art trackers.

A GAUSSIAN PROCESS BASED MULTI-PERSON INTERACTION MODEL

Online multi-person tracking in image sequences is commonly guided by recursive filters, whose predictive models define the expected positions of future states. When a predictive model deviates too much from the true motion of a pedestrian, which is often the case in crowded scenes due to unpredicted accelerations, the data association is prone to fail. In this paper we propose a novel predictive model on the basis of Gaussian Process Regression. The model takes into account the motion of every tracked pedestrian in the scene and the prediction is executed with respect to the velocities of all interrelated persons. As shown by the experiments, the model is capable of yielding more plausible predictions even in the presence of mutual occlusions or missing measurements. The approach is evaluated on a publicly available benchmark and outperforms other state-of-the-art trackers.

Fast two-cycle curve evolution with narrow perception of background for object tracking and contour refinement

The problem of object contour tracking in image sequences remains challenging, especially those with cluttered backgrounds. In this paper, the fast two-cycle level set method with narrow perception of background (FTCNB) is proposed to extract the foreground objects, e.g. vehicles from road image sequences. The curve evolution of the level set method is implemented by computing the signs of region competition terms on two linked lists of contour pixels rather than by solving partial differential equations (PDEs). The curve evolution process mainly consists of two cycles: one cycle for contour pixel evolution and a second cycle for contour pixel smoothness. Based on the curve evolution process, we introduce two tracking stages for the FTCNB method. For coarse tracking stage, the speed function is defined by region competition term combining color and texture features. For contour refinement stage which requires higher tracking accuracy, the likelihood models of the Maximum a posterior (MAP) expressions are incorporated for the speed function. Both the tracking and refinement stages utilize the fast two-cycle curve evolution process with the narrow perception of background regions. With these definitions, we conduct extensive experiments and comparisons for the proposed method. The comparisons with other baseline methods well demonstrate the effectiveness of our work.

Vehicle Association and Tracking in Image Sequences Using Feature-Based Similarity Comparison

Object association and tracking have attracted great attention in the computer vision. In this paper, we present an object association and tracking method for monitoring multiple vehicles on the road based on objects' visual features and the similarity comparison between them. First, we identify vehicles using the difference operation between the current frame in CCTV image sequences and the referential images that are stored in a database, and then extract various features from the vehicles identified. Finally, we associate the objects in the current frame with those in the next frames using similarity comparison, and track multiple objects over a sequence of CCTV image frames. Empirical study using CCTV images shows that our method has achieved the considerable effectiveness in tracking vehicles on the road.

People Tracking using Color Control Points and Skin Color

This paper presents a new approach to be used for people detection and tracking in image sequences based on color control points and skin color modeling. This method aims to track these people in complex situations such as players on a soccer field. Each person in the image is represented by several control points which are obtained using a color version of the Harris algorithm detector. Each control point is characterized by the local appearance which is a vector of local characteristics. Then we determine the rules that define the skin regions in three different color spaces such as RGB, HSV and YCbCr, and we apply these rules to our images to segment skin regions. Using a set of control points and skin regions allows us to track a person by matching control points based on the measure of ZNCC correlation «Zero mean Normalized Cross Correlation». The simulations and experimental results show the robustness of our algorithms in terms of stability and convergence. Performance is illustrated by some examples. Thus, our method fits well with the noise conditions.

Robust Feature Tracking in Image Sequences using View Geometric Constraints

Abstract. Interest point tracking across a sequence of images is a fundamental technique for determining scene motion characteristics. Traditionally, feature tracking has been performed with a variety of appearance-based comparison methods. The most common methods analyze intensity values of local pixels and subsequently attempt to match them to the most similar region in the following frame. This standard, though sometimes effective, lacks versatility. For example, these methods are easily confused by shadows, patterns, feature occlusion, and a variety of other appearance-based anomalies. To counteract the issues presented by a one-sided approach, a new method has been developed to take advantage of both appearance and geometric constraints in a complementary fashion. Using an iterative scheme, camera parameters are computed at each new frame and used to project a derived shape to new point coordinates. This process is repeated for each new frame until a trajectory is created for the entire video sequence. With this method, weight can be allocated as desired between both appearance and geometric constraints. If an issue arises with one constraint (e.g., occlusion or rapid camera movement), the other constraint will continue to track the feature successfully. Experimental results have shown that this method is robust to tracking challenges such as occlusion, shadows, and repeating patterns, while also outperforming appearance-based methods in tracking quality.

Adaptive earth movers distance‐based Bayesian multi‐target tracking

This study describes a complete system for multiple‐target tracking in image sequences. The target appearance is represented as a set of weighted clusters in colour space. This is in contrast to the more typical use of colour histograms to model target appearance. The use of clusters allows a more flexible and accurate representation of the target, which demonstrates the benefits for tracking. However, it also introduces a number of computational difficulties, as calculating and matching cluster signatures are both computationally intensive tasks. To overcome this, the authors introduce a new formulation of incremental medoid‐shift clustering that operates faster than mean shift in multi‐target tracking scenarios. This matching scheme is integrated into a Bayesian tracking framework. Particle filters, a special case of Bayesian filters where the state variables are non‐linear and non‐Gaussian, are used in this study. An adaptive model update procedure is proposed for the cluster signature representation to handle target changes with time. The model update procedure is demonstrated to work successfully on a synthetic dataset and then on real datasets. Successful tracking results are shown on public datasets. Both qualitative and quantitative evaluations have been carried out to demonstrate the improved performance of the proposed multi‐target tracking system. A higher tracking accuracy in long image sequences has been achieved compared to other standard tracking methods.

Efficient Iterative Pose Estimation Using an Invariant to Rotations

This paper deals with pose estimation using an iterative scheme. We show that using adequate visual information, pose estimation can be performed iteratively with only three independent unknowns, which are the translation parameters. Specifically, an invariant to rotational motion is used to estimate the camera position. In addition, an adequate transformation is applied to the proposed invariant to decrease the nonlinearities between the variations in image space and 3-D space. Once the camera position is estimated, we show that the rotation can be estimated efficiently using two different direct methods. The proposed approach is compared against two other methods from the literature. The results show that using our method, pose tracking in image sequences and the convergence rate for randomly generated poses are improved.

Second-Order Optimization of Mutual Information for Real-Time Image Registration

In this paper, we present a direct image registration approach that uses mutual information (MI) as a metric for alignment. The proposed approach is robust and gives an accurate estimation of a set of 2-D motion parameters in real time. MI is a measure of the quantity of information shared by signals. Although it has the ability to perform robust alignment with illumination changes, multimodality, and partial occlusions, few works have proposed MI-based applications related to spatiotemporal image registration or object tracking in image sequences because of some optimization problems, which we will explain. In this paper, we propose a new optimization method that is adapted to the MI cost function and gives a practical solution for real-time tracking. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the registration results are far more robust and accurate than the existing solutions, with the computation also being cheaper. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed.

Coupled Prediction Classification for Robust Visual Tracking

This paper addresses the problem of robust template tracking in image sequences. Our work falls within the discriminative framework in which the observations at each frame yield direct probabilistic predictions of the state of the target. Our primary contribution is that we explicitly address the problem that the prediction accuracy for different observations varies, and in some cases, can be very low. To this end, we couple the predictor to a probabilistic classifier which, when trained, can determine the probability that a new observation can accurately predict the state of the target (that is, determine the "relevance" or "reliability" of the observation in question). In the particle filtering framework, we derive a recursive scheme for maintaining an approximation of the posterior probability of the state in which multiple observations can be used and their predictions moderated by their corresponding relevance. In this way, the predictions of the "relevant" observations are emphasized, while the predictions of the "irrelevant" observations are suppressed. We apply the algorithm to the problem of 2D template tracking and demonstrate that the proposed scheme outperforms classical methods for discriminative tracking both in the case of motions which are large in magnitude and also for partial occlusions.

Registration of real-time X-ray image sequences for weld inspection

Real-time X-ray inspection is one of the important nondestructive testing methods used for long consecutive weld inspection. Registration of the image sequences is the foundation for spatio-temporal image processing methods in which the position relationship between image sequences is needed, such as spatio-temporal filtering, motion blur removing and defect tracking in image sequences. However, the image registration is difficult because the signal-to-noise ratio of real-time X-ray weld image is rather low and the images are quite similar along the weld direction. In this paper, the image registration processing is separated into rotation registration and translation registration. The slope angle of weld, which is calculated by weld edges, is used to implement the rotation registration. And the translation registration is implemented based on phase correlation by extracting features from difference image. The experimental results show that the proposed method is stable and precise.

Object tracking using parallel local colour histogram method

In this paper, a novel algorithm for object tracking in image sequences using local colour histogram method (LCHM) is presented. In order to represent the object to be tracked, the proposed local colour histogram model divides the image into distinct blocks of same size and encodes the colour distribution of each local block. The histogram of each local block of the query image is compared in parallel with the corresponding local block of the test image(s) and the similarity measure is computed using a metric and compared against a threshold. Histogram matching is performed by distance measures like histogram intersection, Euclidean distance and histogram quadratic distance and their performance for detecting the presence of the object in the image is compared. Experimental results show that for retrieval of visually similar object from the image sequences, the local histogram method gives good retrieval precision with speed.

Multiple object detection and tracking with pseudo-particle filter

To tackle the divergence of the classical particle filter method for multiple object tracking in image sequences, a new particle filter, called pseudoparticle filter (PPF), is proposed. The PPF invokes subset particles of generic particle filters to form a continuous estimate of the posterior density function of the objects. After sampling-importance resampling (SIR), the subset particles converge to the observations. It is proved that, using an appropriate kernel function of the mean shift algorithm, we can get the subset particles of the observations and the fixed points of clustering results as the state of the objects. A multiple object data association and state estimation technique is proposed to resolve the subset particles correspondence ambiguities that arise when multiple objects are present. Experimental results demonstrate the efficiency and effectiveness of the algorithm for single and multiple object tracking.

Particle Swarm Optimization Based Object Tracking

This paper proposes a particle swarm optimization based algorithm for object tracking in image sequences. The parametric models of variability of the object appearance are employed to shift the particle swarm in order to cover the promising object location. Afterwards the particles are drawn from a Gaussian distribution. Then the particle swarm optimization takes place in order to concentrate the particles near the true object state. A grayscale appearance model that is learned online is utilized in evaluation of the particles score. Experimental results thatwere obtained in a typical office environment show the feasibility of our approach, especially when the object undergoing tracking has a rapid motion or the appearance changes are considerable. The resulting algorithm runs in real-time on a standard computer.

Partial Linear Gaussian Models for Tracking in Image Sequences Using Sequential Monte Carlo Methods

Partial Linear Gaussian Models for Tracking in Image Sequences Using Sequential Monte Carlo Methods

Partial Linear Gaussian Models for Tracking in Image Sequences Using Sequential Monte Carlo Methods

The recent development of Sequential Monte Carlo methods (also called particle filters) has enabled the definition of efficient algorithms for tracking applications in image sequences. The efficiency of these approaches depends on the quality of the state-space exploration, which may be inefficient due to a crude choice of the function used to sample in the associated probability space. A careful study of this issue led us to consider the modeling of the tracked dynamic system with partial linear Gaussian models. Such models are characterized by a non linear dynamic equation, a linear measurement equation and additive Gaussian noises. They allow inferring an analytic expression of the optimal importance function used in the diffusion process of the particle filter, and enable building a relevant approximation of a validation gate. Despite of these potential advantages partial linear Gaussian models have not been investigated. The aim of this paper is therefore to demonstrate that such models can be of real interest facing difficult usual issues such as occlusions, ambiguities due to cluttered backgrounds and large state space. Three instances of these models are proposed. After a theoretical analysis, their significance is demonstrated by their performance for tracking points and planar objects in challenging real-world image sequences.