Multi-target Visual Tracking Research Articles

Multi-target trajectory tracking in multi-frame video images of basketball sports based on deep learning

INTRODUCTION: There is occlusion interference in the multi-target visual tracking process of basketball video images, which leads to poor accuracy of multi-target trajectory tracking. This paper studies the multi-target trajectory tracking method in multi-frame video images of basketball sports based on deep learning. OBJECTIVES: Aiming at the problem of target occlusion in the tracking process and the problem of trajectory tracking anomaly caused by target occlusion, a modified algorithm is proposed. METHODS: The method is divided into two parts: detection and tracking. In the detection part, the YOLOv3 algorithm in deep learning technology is used to detect each target in the video, and the original YOLOv3 backbone network Darknet-53 is replaced by the lightweight backbone network MobileNetV2 to extract the target features. RESULTS: Based on the target detection results, the Kalman filter is used to predict the next position and bounding box size of the target to obtain the target trajectory prediction results according to the current target position, then a hierarchical data association algorithm is designed, and multi-target tracking of the same category is completed based on the target appearance feature similarity and feature similarity. CONCLUSION: The experimental results show that the method can accurately detect the targets in multi-frame video images in basketball sports and obtain high-precision target trajectory tracking results.

Robust online multi-target visual tracking using a HISP filter with discriminative deep appearance learning

We propose a novel online multi-target visual tracker based on the recently developed Hypothesized and Independent Stochastic Population (HISP) filter. The HISP filter combines advantages of traditional tracking approaches like MHT and point-process-based approaches like PHD filter, and it has linear complexity while maintaining track identities. We apply this filter for tracking multiple targets in video sequences acquired under varying environmental conditions and targets density using a tracking-by-detection approach. We also adopt deep CNN appearance representation by training a verification-identification network (VerIdNet) on large-scale person re-identification data sets. We construct an augmented likelihood in a principled manner using this deep CNN appearance features and spatio-temporal information. Furthermore, we solve the problem of two or more targets having identical label considering the weight propagated with each confirmed hypothesis. Extensive experiments on MOT16 and MOT17 benchmark data sets show that our tracker significantly outperforms several state-of-the-art trackers in terms of tracking accuracy.

An iterative pose estimation algorithm based on epipolar geometry with application to multi-target tracking

This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit &#x0028 IMU &#x0029 is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking &#x0028 MTT &#x0029 in the presence of parallax and demonstrate its real-time performance on a 640 &#x00D7 480 video sequence captured on a UAV.

On-Line Visual Tracking with Occlusion Handling.

One of the core challenges in visual multi-target tracking is occlusion. This is especially important in applications such as video surveillance and sports analytics. While offline batch processing algorithms can utilise future measurements to handle occlusion effectively, online algorithms have to rely on current and past measurements only. As such, it is markedly more challenging to handle occlusion in online applications. To address this problem, we propagate information over time in a way that it generates a sense of déjà vu when similar visual and motion features are observed. To achieve this, we extend the Generalized Labeled Multi-Bernoulli (GLMB) filter, originally designed for tracking point-sized targets, to be used in visual multi-target tracking. The proposed algorithm includes a novel false alarm detection/removal and label recovery methods capable of reliably recovering tracks that are even lost for a substantial period of time. We compare the performance of the proposed method with the state-of-the-art methods in challenging datasets using standard visual tracking metrics. Our comparisons show that the proposed method performs favourably compared to the state-of-the-art methods, particularly in terms of ID switches and fragmentation metrics which signifies occlusion.

Box Particle Probability Hypothesis Density Filter for Multi-target Visual Tracking

Box Particle Probability Hypothesis Density Filter for Multi-target Visual Tracking

A Robust Multi-Target Visual Tracking Method using TLD Framework

This paper investigates a robust multi-target visual tracking method based on TLD framework. We propose a novel visual tracking method that divides the long-term tracking task into four parts: tracker module, learning module, weak-detector module and strong-detector module. Before the tracking, it is essential to sample some basic posture of targets and build several patterns by training. Then, a separate process is allocated for each target may appear in the video or not. The strong-detector module makes a global scanning on every frame. Once a target of interest is detected, the tracker module starts to follow it from frame to frame. The weak-detector module makes a local scanning around the position predicted by the tracker module. The learning module estimates the errors of weak-detector module and updates the weak-detector module to avoid these errors from coming up again. At the last, when the posture of some target has large change, it may result in the metal failure of tracker module and weak-detector module. At this time, the strong-detector module will relocate the target and initialize the others modules.

Tracking multiple targets based on min-cost network flows with detection in RGB-D data

Visual multi-target tracking technology is a challenging problem in computer vision. This study proposes a novel approach for multi-target tracking based on min-cost network flows in RGB-D data with tracking-by-detection scheme. Firstly, the moving objects are detected by fusing RGB information and depth information. Then, we formulate the multi-target tracking problem as a maximum a posteriori (MAP) estimation problem with specific constraints, and the problem is converted into a cost-flow network. Finally, using a min-cost flow algorithm, we can obtain the tracking results. Extensive experimental results show that the proposed algorithm greatly improves the robustness and accuracy and outperforms the state-of-the-art significantly.

GM-PHD-Based Multi-Target Visual Tracking Using Entropy Distribution and Game Theory

Tracking multiple moving targets in a video is a challenge because of several factors, including noisy video data, varying number of targets, and mutual occlusion problems. The Gaussian mixture probability hypothesis density (GM-PHD) filter, which aims to recursively propagate the intensity associated with the multi-target posterior density, can overcome the difficulty caused by the data association. This paper develops a multi-target visual tracking system that combines the GM-PHD filter with object detection. First, a new birth intensity estimation algorithm based on entropy distribution and coverage rate is proposed to automatically and accurately track the newborn targets in a noisy video. Then, a robust game-theoretical mutual occlusion handling algorithm with an improved spatial color appearance model is proposed to effectively track the targets in mutual occlusion. The spatial color appearance model is improved by incorporating interferences of other targets within the occlusion region. Finally, the experiments conducted on publicly available videos demonstrate the good performance of the proposed visual tracking system.

Entropy distribution and coverage rate-based birth intensity estimation in GM-PHD filter for multi-target visual tracking

Tracking multiple moving targets in video is a challenge because of the presence of noisy video data and varying numbers of targets, and data association problems. In this paper, a multi-target visual tracking system that combines object detection with the Gaussian mixture probability hypothesis density filter is developed, in which a new birth intensity estimation method based on entropy distribution and coverage rate is proposed. The birth intensity is first initialized by the previously obtained target states and measurements. The measurements are obtained by object detection and are classified into the birth measurements and the survival measurements. The currently obtained birth measurements are then used to update the birth intensity. In the update stage, the entropy distribution is incorporated to remove some noises within the initialized birth intensity that are irrelevant to the birth measurements. The coverage rate between each birth intensity component and the corresponding birth measurement is computed to further eliminate the noises. Experiments on noisy video sequences are conducted to show the good performance of the proposed visual tracking system.

Particle Probability-hypothesis-density Filter with Kernel Based State Extraction for Efficient Multi-target Visual Tracking

Particle Probability-hypothesis-density Filter with Kernel Based State Extraction for Efficient Multi-target Visual Tracking

Multi-target visual tracking and occlusion detection by combining Bhattacharyya Coefficient and Kalman filter innovation

This paper introduces an approach for visual tracking of multi-target with occlusion occurrence. Based on the author’s previous work in which the Overlap Coefficient (OC) is used to detect the occlusion, in this paper a method of combining Bhattacharyya Coefficient (BC) and Kalman filter innovation term is proposed as the criteria for jointly detecting the occlusion occurrence. Fragmentation of target is introduced in order to closely monitor the occlusion development. In the course of occlusion, the Kalman predictor is applied to determine the location of the occluded target, and the criterion for checking the re-appearance of the occluded target is also presented. The proposed approach is put to test on a standard video sequence, suggesting the satisfactory performance in multi-target tracking.

Game-theoretical occlusion handling for multi-target visual tracking

Multi-target visual tracking is a challenge because of data association and mutual occlusion in the interacting targets. This paper presents a Gaussian mixture probability hypothesis density based multi-target visual tracking system with game-theoretical occlusion handling. Firstly, the spatial constraint based appearance model with other interacting targets’ interferences is modeled. Then, a two-step occlusion reasoning algorithm is proposed. Finally, an n-person, non-zero-sum, non-cooperative game is constructed to handle the mutual occlusion problem. The individual targets within the occlusion region are regarded as the players in the constructed game to compete for the maximum utilities by using the certain strategies. A Nash Equilibrium of the game is the optimal estimation of the locations of the players within the occlusion region. Experiments on video sequences demonstrate the good performance of the proposed occlusion handling algorithm.

An on-line learning method for face association in personal photo collection

Due to the widespread use of cameras, it is very common to collect thousands of personal photos. A proper organization is needed to make the collection usable and to enable an easy photo retrieval. In this paper, we present a method to organize personal photo collections based on “who” is in the picture. Our method consists in detecting the faces in the photo sequence and arranging them in groups corresponding to the probable identities. This problem can be conveniently modeled as a multi-target visual tracking where a set of on-line trained classifiers is used to represent the identity models. In contrast to other works where clustering methods are used, our method relies on a probabilistic framework; it does not require any prior information about the number of different identities in the photo album. To enable future comparison, we present experimental results on a public dataset and on a photo collection generated from a public face dataset.

Particle probability hypothesis density filtering for multitarget visual tracking with robust state extraction

Particle probability hypothesis density (PHD) filter-based visual trackers have achieved considerable success in the visual tracking field. But position measurements based on detection may not have enough ability to discriminate an object from clutter, and accurate state extraction cannot be obtained in the original PHD filtering framework, especially when targets can appear, disappear, merge, or split at any time. To meet the limitations, the proposed algorithm combines a color histogram of a target and the temporal dynamics in a unifying framework and a Gaussian mixture model clustering method for efficient state extraction is designed. The proposed tracker can improve the accuracy of state estimation in tracking a variable number of objects.

Probability-hypothesis-density filter for multitarget visual tracking with trajectory recognition

The probability-hypothesis-density (PHD) filter as a multitarget recursive Bayes filter has generated substantial interest in the visual tracking field due to its ability to handle a time-varying number of targets. But the target's trajectory cannot be identified within its own framework. To complement the ability of PHD, the auction algorithm is combined to calculate the object trajectories automatically. We present a motion detection, dynamic, and measurement equation, as well as visual multitarget tracking algorithm based on Gaussian mixture probability hypothesis density with trajectory computation in detail. Experimental results on a large video surveillance dataset show that the proposed multitarget tracking framework improves the tracker and recognizes tracks when a variable number of targets appear, merge, split, and disappear, even in cluttered scenes.

A real-time multitarget tracking system with robust multichannel CNN-UM algorithms

This paper introduces a tightly coupled topographic sensor-processor and digital signal processor (DSP) architecture for real-time visual multitarget tracking (MTT) applications. We define real-time visual MTT as the task of tracking targets contained in an input image flow at a sampling-rate that is higher than the speed of the fastest maneuvers that the targets make. We utilize a sensor-processor based on the cellular neural network universal machine architecture that permits the offloading of the main image processing tasks from the DSP and introduces opportunities for sensor adaptation based on the tracking performance feedback from the DSP. To achieve robustness, the image processing algorithms running on the sensor borrow ideas from biological systems: the input is processed in different parallel channels (spatial, spatio-temporal and temporal) and the interaction of these channels generates the measurements for the digital tracking algorithms. These algorithms (running on the DSP) are responsible for distance calculation, state estimation, data association and track maintenance. The performance of the proposed system is studied using actual hardware for different video flows containing rapidly moving maneuvering targets.