Multi-object Tracking Method Research Articles

Deep Continuous Conditional Random Fields With Asymmetric Inter-Object Constraints for Online Multi-Object Tracking

Online Multi-Object Tracking (MOT) is a challenging problem and has many important applications including intelligence surveillance, robot navigation and autonomous driving. In existing MOT methods, individual object's movements and inter-object relations are mostly modeled separately and relations between them are still manually tuned. In addition, inter-object relations are mostly modeled in a symmetric way, which we argue is not an optimal setting. To tackle those difficulties, in this paper, we propose a Deep Continuous Conditional Random Field (DCCRF) for solving the online MOT problem in a track-by-detection framework. The DCCRF consists of unary and pairwise terms. The unary terms estimate tracked objects' displacements across time based on visual appearance information. They are modeled as deep Convolution Neural Networks, which are able to learn discriminative visual features for tracklet association. The asymmetric pairwise terms model inter-object relations in an asymmetric way, which encourages high-confidence tracklets to help correct errors of low-confidence tracklets and not to be affected by low-confidence ones much. The DCCRF is trained in an end-to-end manner for better adapting the influences of visual information as well as inter-object relations. Extensive experimental comparisons with state-of-the-arts as well as detailed component analysis of our proposed DCCRF on two public benchmarks demonstrate the effectiveness of our proposed MOT framework.

An algorithm to track laboratory zebrafish shoals

In this paper, a semi-automatic multi-object tracking method to track a group of unmarked zebrafish is proposed. This method can handle partial occlusion cases, maintaining the correct identity of each individual. For every object, we extracted a set of geometric features to be used in the two main stages of the algorithm. The first stage selected the best candidate, based both on the blobs identified in the image and the estimate generated by a Kalman Filter instance. In the second stage, if the same candidate-blob is selected by two or more instances, a blob-partitioning algorithm takes place in order to split this blob and reestablish the instances' identities. If the algorithm cannot determine the identity of a blob, a manual intervention is required.This procedure was compared against a manual labeled ground truth on four video sequences with different numbers of fish and spatial resolution. The performance of the proposed method is then compared against two well-known zebrafish tracking methods found in the literature: one that treats occlusion scenarios and one that only track fish that are not in occlusion. Based on the data set used, the proposed method outperforms the first method in correctly separating fish in occlusion, increasing its efficiency by at least 8.15% of the cases. As for the second, the proposed method's overall performance outperformed the second in some of the tested videos, especially those with lower image quality, because the second method requires high-spatial resolution images, which is not a requirement for the proposed method. Yet, the proposed method was able to separate fish involved in occlusion and correctly assign its identity in up to 87.85% of the cases, without accounting for user intervention.

Online multiple objects tracking with detection reliability prior constraint

Multi-object tracking (MOT) is one popular topic in computer vision. It remains a challenging problem in complex scenes, especially of objects with similar appearance. In this case, many existing data association strategies, which link detections among consecutive frames according appearance and motion cues, may fail to track due to unreliable detections or confused appearance and motion. To solve this problem, this paper proposed a novel online multi-object tracking method with detection reliability prior constraint. Our method integrates the trajectory estimation and detection-prediction association into a unified framework. The detection reliability prior constraint is built with the Hankel matrix from object motion model. When we build the Hankel matrix, we adaptively select a set of previous frames to predict object states and calculate the associated weights between detections and candidate objects. Data association in MOT then is estimated by maximum a posteriori (MAP) in a Bayesian framework, accompanied with both previous trajectory and the current detection reliability. Experimental results using synthetic dataset and four public challenging datasets demonstrate that, the proposed method has a good tracking performance compared with the state-of-the-art multi-object trackers.

Learning Discriminative Appearance Models for Online Multi-Object Tracking With Appearance Discriminability Measures

A data association, which links detections in consecutive frames, is a key issue in multi-object tracking (MOT). For robust data association in a complex scene, a common approach is to learn object appearance models for handling appearance variations of tracked objects and improving the discriminability between objects. However, learning appearances of multiple objects during tracking is still a challenging problem due to the frequent sample contamination by occlusions and low feature discriminability by similar appearances between objects. In this paper, in order to learn each object appearance, we propose a discriminative online appearance learning using a partial least square (PLS) method. In the proposed appearance learning, we first present online sampling mining to collect training samples from tracking results. Then, we consecutively learn PLS-based subspaces during tracking and discriminate object appearances by projecting object features onto the learned spaces. Since frequent appearance updates for all tracked objects increase the tracking complexity significantly, we propose measures to evaluate the discriminability of learned object appearances and update only the appearances with low discriminability. We apply the proposed appearance learning for online MOT and compare other appearance learning methods. In addition, we evaluate the performance of our MOT method on public MOT benchmark challenge datasets and show the competitive performance compared to other state-of-the-art batch and online tracking methods.

Online Multi-Object Tracking Using Hierarchical Constraints for Complex Scenarios

Online multi-object tracking (MOT) in an intelligent vehicle platform aims at locating the surrounding objects in real time, which remains far from being solved in complex scenarios, due to various motion patterns of tracked objects and severe occlusions caused by cluttered background or other objects. In this paper, we establish a unified online MOT framework for complex scenarios that employs a hierarchical model to improve the solution of data association, termed hierarchical MOT (HMOT). Incorporating the multiple Gaussians uncertainty theory into the individual motion model for each target followed by imposing interaction constraint to re-associate the tracklets with lower confidence leads our algorithm to achieve accurate multi-object tracking. With such a model, individual objects are not only more precisely associated across frames, but also dynamically constrained with each other in a global manner. Experiments on challenging data sets verify the performance of the proposed HMOT approach over the other state-of-the-art MOT methods.

A Trajectory Based Method of Automatic Counting of Cyclist in Traffic Video Data

Due to the growing number of cyclist accidents on urban roads, methods for collecting information on cyclists are of significant importance to the Department of Transportation. The collected information provides insights into solving critical problems related to transportation planning, implementing safety countermeasures, and managing traffic flow efficiently. Intelligent Transportation System (ITS) employs automated tools to collect traffic information from traffic video data. One of the important factors that influence cyclists safety is their counts. In comparison to other road users, such as cars and pedestrians, the automated cyclist data collection is relatively a new research area. In this work, we develop a vision-based method for gathering cyclist count data at intersections and road segments. We implement a robust cyclist detection method based on a combination of classification features. We implement a multi-object tracking method based on the Kernelized Correlation Filters (KCF) in cooperation with the bipartite graph matching algorithm to track multiple cyclists. Then, a trajectory rebuilding method and a trajectory comparison model are applied to refine the accuracy of tracking and counting. The proposed method is the first cyclist counting method, that has the ability to count cyclists under different movement patterns. The trajectory data obtained can be further utilized for cyclist behavioral modeling and safety analysis.

Confidence-Based Data Association and Discriminative Deep Appearance Learning for Robust Online Multi-Object Tracking.

Online multi-object tracking aims at estimating the tracks of multiple objects instantly with each incoming frame and the information provided up to the moment. It still remains a difficult problem in complex scenes, because of the large ambiguity in associating multiple objects in consecutive frames and the low discriminability between objects appearances. In this paper, we propose a robust online multi-object tracking method that can handle these difficulties effectively. We first define the tracklet confidence using the detectability and continuity of a tracklet, and decompose a multi-object tracking problem into small subproblems based on the tracklet confidence. We then solve the online multi-object tracking problem by associating tracklets and detections in different ways according to their confidence values. Based on this strategy, tracklets sequentially grow with online-provided detections, and fragmented tracklets are linked up with others without any iterative and expensive association steps. For more reliable association between tracklets and detections, we also propose a deep appearance learning method to learn a discriminative appearance model from large training datasets, since the conventional appearance learning methods do not provide rich representation that can distinguish multiple objects with large appearance variations. In addition, we combine online transfer learning for improving appearance discriminability by adapting the pre-trained deep model during online tracking. Experiments with challenging public datasets show distinct performance improvement over other state-of-the-arts batch and online tracking methods, and prove the effect and usefulness of the proposed methods for online multi-object tracking.

Online multi-object tracking with efficient track drift and fragmentation handling.

This paper addresses the problem of multi-object tracking in complex scenes by a single, static, uncalibrated camera. Tracking-by-detection is a widely used approach for multi-object tracking. Challenges still remain in complex scenes, however, when this approach has to deal with occlusions, unreliable detections (e.g., inaccurate position/size, false positives, or false negatives), and sudden object motion/appearance changes, among other issues. To handle these problems, this paper presents a novel online multi-object tracking method, which can be fully applied to real-time applications. First, an object tracking process based on frame-by-frame association with a novel affinity model and an appearance update that does not rely on online learning is proposed to effectively and rapidly assign detections to tracks. Second, a two-stage drift handling method with novel track confidence is proposed to correct drifting tracks caused by the abrupt motion change of objects under occlusion and prolonged inaccurate detections. In addition, a fragmentation handling method based on a track-to-track association is proposed to solve the problem in which an object trajectory is broken into several tracks due to long-term occlusions. Based on experimental results derived from challenging public data sets, the proposed method delivers an impressive performance compared with other state-of-the-art methods. Furthermore, additional performance analysis demonstrates the effect and usefulness of each component of the proposed method.

A multiCell visual tracking algorithm using multi-task particle swarm optimization for low-contrast image sequences

In terms of the varying number of cell population, shape deformation, collision and uneven movement, a novel method based on multi-task particle swarm optimization (PSO) algorithm without explicit detection module, named MTPSO tracking method, is developed for automatic tracking of biological cells in time-lapse low-contrast microscopy image sequences. For tracking existing cells from the previous frames, a PSO-based tracking module is firstly implemented to give the initial positions of existing cells according to the previous estimated state of each cell, then a PSO-based contour module is proposed to determine the corresponding contour of each cell and finally achieve a precise position tracking by an iterative centroid updating process. For tracking new appearing cells at the current frame, a PSO-based discovery module, followed by the aforementioned PSO-based contour module, is proposed to search for new potential cells through appropriate initialization of particle swarm and searching mechanism. MTPSO tracking method is tested over a number of different real cell image sequences and is shown to provide high accuracy both in position and contour estimate of each cell in various challenging cases. Furthermore, it is more competitive against the state-of-the-art multi-object tracking methods in terms of performance measures such as FAR, FNR, LTR, and LSR.

A literature review on recent multi-object tracking methods based on HMM and particle filter

Tracking of the object movement in the input frame of video is an important process for various real time application such as video-conferencing, human robotics or human computer interface or in the analysis of social interaction. The important step is to determine the path of the object. The various techniques to track the multiple moving objects in a input frame of video have been proposed. This paper gives a brief analysis of recent multi-object trackers algorithms based on markov model and particle filtering

Data Association Based Tracking Traffic Objects

For the widely demanding of adaptive multiple moving objects tracking in intelligent transportation field, a new type of traffic video based multi-object tracking method is presented. Background is modeled by difference of Gaussians (DOG) probability kernel and background subtraction is used to detect multiple moving objects. After obtaining the foreground, shadow is eliminated by an edge detection method. A type of particle filtering combined with SIFT method is used for motion tracking. A queue chain method is used to record data association among different objects, which could improve the detection accuracy and reduce the complexity. By actual road tests, the system tracks multi-object with a better performance of real time and mutual occlusion robustness, indicating that it is effective for intelligent transportation system.

A hybrid tracking method for scaled and oriented objects in crowded scenes

Traditional kernel based means shift assumes constancy of the object scale and orientation during the course of tracking and uses a symmetric/asymmetric kernel, such as a circle or an ellipse for target representation. In a tracking scenario, it is not uncommon to observe objects with complex shapes whose scale and orientation constantly change due to the camera and object motions. In this paper, we propose a multi object tracking method which tracks the complete object regions, adapts to changing scale and orientation, and assigns consistent labels to each object throughout real world video sequences. Our approach has five major components: (1) dynamic background subtraction, (2) level sets, (3) mean shift convergence, (4) object identification, and (5) occlusion handling. The experimental results show that the proposed method is superior to the traditional mean shift tracking in the following aspects: (1) it provides consistent multi objects tracking instead of single object throughout the video, (2) it is not affected by the scale and orientation changes of the tracked objects, (3) its computational complexity is much less than traditional mean shift due to using level set method instead of probability density.

Information fusion for automotive applications – An overview

This article focusses on the fusion of information from various automotive sensors like radar, video, and lidar for enhanced safety and traffic efficiency. Fusion is not restricted to data from sensors onboard the same vehicle but vehicular communication systems allow to propagate and fuse information with sensor data from other vehicles or from the road infrastructure as well. This enables vehicles to perceive information from regions that are hardly accessible otherwise and represents the basis for cooperative driving maneuvers. While the Bayesian framework builds the basis for information fusion, automobile environments are characterized by their a priori unknown topology, i.e., the number, type, and structure of the perceived objects is highly variable. Multi-object detection and tracking methods are a first step to cope with this challenge. Obviously, the existence or non-existence of an object is of paramount importance for safe driving. Such decisions are highly influenced by the association step that assigns sensor measurements to object tracks. Methods that involve multiple sequences of binary assignments are compared with soft-assignment strategies. Finally, fusion based on finite set statistics that (theoretically) avoid an explicit association are discussed.

Tracking video objects with feature points based particle filtering

For intelligent video surveillance, the adaptive tracking of multiple moving objects is still an open issue. In this paper, a new multi-object tracking method based on video frames is proposed. A type of particle filtering combined with the SIFT (Scale Invariant Feature Transform) is proposed for motion tracking, where SIFT key points are treated as parts of particles to improve the sample distribution. Then, a queue chain method is adopted to record data associations among different objects, which could improve the detection accuracy and reduce the computational complexity. By actual road tests and comparisons, the system tracks multi-objects with better performance, e.g., real time implementation and robust against mutual occlusions, indicating that it is effective for intelligent video surveillance systems.

Human-following robot using the particle filter in ISpace with distributed vision sensors

We present a method for representing tracking and human-following by fusing distributed multiple vision systems in intelligent space, with applications to pedestrian tracking in a crowd. In this context, particle filters provide a robust tracking framework under ambiguous conditions. The particle filter technique is used in this work, but in order to reduce its computational complexity and increase its robustness, we propose to track the moving objects by generating hypotheses not in the image plan but on a top-view reconstruction of the scene. Comparative results on real video sequences show the advantage of our method for multiobject tracking. Simulations are carried out to evaluate the proposed performance. Also, the method is applied to the intelligent environment, and its performance is verified by experiments.

Multi-Object Tracking using the Color-Based Particle Filter in ISpace with Distributed Sensor Network

Intelligent Space(ISpace) is the space where many intelligent devices, such as computers and sensors, are distributed. According to the cooperation of many intelligent devices, the environment, it is very important that the system knows the location information to offer the useful services. In order to achieve these goals, we present a method for representing, tracking and human following by fusing distributed multiple vision systems in ISpace, with application to pedestrian tracking in a crowd. And the article presents the integration of color distributions into particle filtering. Particle filters provide a robust tracking framework under ambiguity conditions. We propose to track the moving objects by generating hypotheses not in the image plan but on the top-view reconstruction of the scene. Comparative results on real video sequences show the advantage of our method for multi-object tracking. Simulations are carried out to evaluate the proposed performance. Also, the method is applied to the intelligent environment and its performance is verified by the experiments.