Multi-object Tracking Algorithm Research Articles

STN-Track: Multiobject Tracking of Unmanned Aerial Vehicles by Swin Transformer Neck and New Data Association Method

The gradual development of remote sensing object tracking technology based on unmanned aerial vehicles (UAV) videos has become one of the main research directions in the field of visual tracking. However, due to characteristics of the UAV platform, typical visual tracking algorithms currently applied to natural scenes cannot be used directly. Small-scale objects in UAV remote sensing videos are difficult to detect and have the problem of tracking identity switching. In order to solve these problems, we designed the Swin transformer neck YOLOX (STN-YOLOX) object detection algorithm as the detection module, and the G-Byte data association method as the tracking module. We then combined the two into a new multi-object tracking algorithm named STN-Track. We used STN-Track to conduct experiments on the UAVDT and VisDrone MOT datasets. The experimental results show that compared with the current state-of-the-art (SOTA) methods, our STN-Track has improved detection and tracking accuracy of small-scale objects and greatly improved identification capabilities for object tracking. Compared with the SOTA ByteTrack algorithm, MOTA of STN-Track can be improved by up to 3.2%, AP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</sub> can be improved by up to 4.4%, MT can be improved by up to 6.8%, and IDSW can be reduced by up to 28.0%.

Multivehicle Object Tracking in Satellite Video Enhanced by Slow Features and Motion Features

With the development of video satellites, multimoving object tracking in satellite video is possible and has become a new challenging task. The difficulties are mainly caused by the characteristics of satellite videos: 1) small objects; 2) low contrast between objects and background; and 3) background in a state of continuous motion. These characteristics make it difficult for the advanced multiobject tracking algorithms in the natural video to give full play to their advantages, resulting in vast false alarms, missed objects, ID switches, and low-confidence bounding boxes. To tackle these problems, a novel multimoving object tracking method considering slow features (SFs) and motion features has been proposed in this research, named SF and motion feature-guided multiobject tracking (SFMFMOT), which realizes the continuous tracking of moving vehicles in satellite videos. A nonmaximum suppression (NMS) module guided by bounding box proposals based on SFs is designed to assist the object detection part by utilizing the sensitivity of SF analysis to the changed pixels. While removing a large number of static false alarms and supplementing missed objects, it improves the recall rate by increasing the confidence score of the correctly detected object bounding boxes. In order to improve the tracking performance, a set of optimization strategies based on motion features and time accumulation information are proposed to smooth the trajectory, remove static false alarms, and duplicate bounding boxes. The proposed method is evaluated in three satellite videos and its superiority is demonstrated.

A PERUSAL STUDY OF METHODS FOR ENERGY EFFICIENT MULTI OBJECT TRACKING IN WIRELESS SENSOR NETWORKS

Wireless sensor networks (WSNs) has various applications in different areas such as object detection and target tracking, industrial process monitoring, environmental monitoring, wildlife monitoring, Robot control and security applications for buildings. In WSNs, Energy efficiency is one of the important research issues, as it determines the lifetime of the sensor network deployed for the specific applications. Object tracking is one of the dominant applications of wireless sensor networks and many energy-efficient multi object tracking algorithms are used for precise or exact object tracking. The main focus this paper is the perusal study of all different energy-efficient object tracking methods and algorithms for Wireless Sensor Network.

Data association algorithm for large-scale multi-object tracking with complex interactions

We present an online multi-object tracking algorithm to track multiple objects across a large number of image frames. Our work is motivated by the need to study evolution of nanoscale objects by transmission electron microscopy. The proposed approach is based on the existing multi-way data association tracking algorithm that is capable of tracking interacting objects with complex behaviors (i.e., merge, split, overlap, and appearance or disappearance). The multi-way data association is an offline algorithm to associate objects across all image frames at one step with a global optimization, which does not scale very well for large number of image frames. The proposed online tracking algorithm processes image frames as they arrive by detecting all objects in the newly arrived image frame and making the associations of the objects to those detected from the previous frame by the multi-way data association. This frame-by-frame association scheme can cause fragmented traces of the objects that are occasionally misdetected for some image frames. We overcome this issue by allowing previously unassociated objects to be associated when the objects reappear within a fixed number of future image frames, namely the frame-delayed association. We combine the multi-way data association with the frame-delayed association to be able to track interacting objects with accurate handling of object disappearance events. The proposed method is validated through applications to simulated multi-object tracking problem and a real multi-object tracking problem. The outcome of the proposed method is compared with four state-of-the-art algorithms.

Application Research on Roller Skater Detection, Tracking, and Trajectory Prediction Based on Video Stream

With the continuous progress of science and technology, the sport of roller skating has developed rapidly and the technical level of the game has become higher and higher. Its sport performance has been rapidly improved. However, China’s roller skating is relatively late, and there is still a certain gap compared with many Western developed countries. In order to improve the performance of China’s roller skating, this study takes the representative Chinese and foreign excellent speed skaters as the research object and compares the sprinting technology of Chinese and foreign excellent speed skaters by using image measurement and image analysis to obtain the kinematic parameters and data of the athletes’ sprinting technology in the competition state. In view of the problem that the current video target tracking algorithm is easy to follow multiple targets, a video multiobject detection and tracking algorithm with improved tracking learning detection (TLD) is studied with the skater in the video as the research object. For the lost target, the prediction function of Kalman filter algorithm is used to track the trajectory of the typical target in the video, and the trajectory tracked by Kalman filter algorithm is used to compensate the lost part of TLD algorithm, so as to obtain the complete trajectory of the typical target in the video to improve the accuracy of video multiobject tracking. Since the existing trajectory prediction algorithms have the limitation of poor accuracy, a social-long short-term memory (Social-LSTM) network-based video typical target trajectory prediction algorithm is proposed to predict the trajectory sequences of typical targets to be detected by incorporating the contextual environment information and the interaction relationship between multiple target trajectories into the Social-LSTM network. The simulation results show that the proposed trajectory prediction algorithm outperforms the traditional LSTM algorithm, Hidden Markov Model Algorithm, and Hybrid Gaussian model algorithm, which is helpful to improve the accuracy of video roller skater target trajectory prediction, and the tracking success rate is 0.98.

Automatic discard registration in cluttered environments using deep learning and object tracking: class imbalance, occlusion, and a comparison to human review

Abstract This paper presents and evaluates a method for detecting and counting demersal fish species in complex, cluttered, and occluded environments that can be installed on the conveyor belts of fishing vessels. Fishes on the conveyor belt were recorded using a colour camera and were detected using a deep neural network. To improve the detection, synthetic data were generated for rare fish species. The fishes were tracked over the consecutive images using a multi-object tracking algorithm, and based on multiple observations, the fish species was determined. The effect of the synthetic data, the amount of occlusion, and the observed dorsal or ventral fish side were investigated and a comparison with human electronic monitoring (EM) review was made. Using the presented method, a weighted counting error of 20% was achieved, compared to a counting error of 7% for human EM review on the same recordings.

SORT-YM: An Algorithm of Multi-Object Tracking with YOLOv4-Tiny and Motion Prediction

Multi-object tracking (MOT) is a significant and widespread research field in image processing and computer vision. The goal of the MOT task consists in predicting the complete tracklets of multiple objects in a video sequence. There are usually many challenges that degrade the performance of the algorithm in the tracking process, such as occlusion and similar objects. However, the existing MOT algorithms based on the tracking-by-detection paradigm struggle to accurately predict the location of the objects that they fail to track in complex scenes, leading to tracking performance decay, such as an increase in the number of ID switches and tracking drifts. To tackle those difficulties, in this study, we design a motion prediction strategy for predicting the location of occluded objects. Since the occluded objects may be legible in earlier frames, we utilize the speed and location of the objects in the past frames to predict the possible location of the occluded objects. In addition, to improve the tracking speed and further enhance the tracking robustness, we utilize efficient YOLOv4-tiny to produce the detections in the proposed algorithm. By using YOLOv4-tiny, the tracking speed of our proposed method improved significantly. The experimental results on two widely used public datasets show that our proposed approach has obvious advantages in tracking accuracy and speed compared with other comparison algorithms. Compared to the Deep SORT baseline, our proposed method has a significant improvement in tracking performance.

Multiple Ship Tracking in Remote Sensing Images Using Deep Learning

In remote sensing images, small target size and diverse background cause difficulty in locating targets accurately and quickly. To address the lack of accuracy and inefficient real-time performance of existing tracking algorithms, a multi-object tracking (MOT) algorithm for ships using deep learning was proposed in this study. The feature extraction capability of target detectors determines the performance of MOT algorithms. Therefore, you only look once (YOLO)-v3 model, which has better accuracy and speed than other algorithms, was selected as the target detection framework. The high similarity of ship targets will cause poor tracking results; therefore, we used the multiple granularity network (MGN) to extract richer target appearance information to improve the generalization ability of similar images. We compared the proposed algorithm with other state-of-the-art multi-object tracking algorithms. Results show that the tracking accuracy is improved by 2.23%, while the average running speed is close to 21 frames per second, meeting the needs of real-time tracking.

Online multi-object tracking using multi-function integration and tracking simulation training

Recently, with the development of deep-learning, the performance of multi-object tracking algorithms based on deep neural networks has been greatly improved. However, most methods separate different functional modules into multiple networks and train them independently on specific tasks. When these network modules are used directly, they are not compatible with each other effectively, nor can they be better adapted to the multi-object tracking task, which leads to a poor tracking effect. Therefore, a network structure is designed to aggregate the regression of objects between frames and the extraction of appearance features into one model to improve the harmony between various functional modules of multi-object tracking. To improve the support for the multi-object tracking task, an end-to-end training method is also proposed to simulate the multi-object tracking process during the training and expand the training data by using the historical position of the target combined with the prediction of the motion model. A metric loss that can take advantage of the historical appearance features of the target is also used to train the extraction module of appearance features to improve the temporal correlation of extracted appearance features. Evaluation results on the MOTChallenge benchmark datasets show that the proposed approach achieves state-of-the-art performance.

Performance Study of Distance-Weighting Approach with Loopy Sum-Product Algorithm for Multi-Object Tracking in Clutter.

In this paper, we explore the performance of the distance-weighting probabilistic data association (DWPDA) approach in conjunction with the loopy sum-product algorithm (LSPA) for tracking multiple objects in clutter. First, we discuss the problem of data association (DA), which is to infer the correspondence between targets and measurements. DA plays an important role when tracking multiple targets using measurements of uncertain origin. Second, we describe three methods of data association: probabilistic data association (PDA), joint probabilistic data association (JPDA), and LSPA. We then apply these three DA methods for tracking multiple crossing targets in cluttered environments, e.g., radar detection with false alarms and missed detections. We are interested in two performance metrics: tracking accuracy and computation time. LSPA is known to be superior to PDA in terms of the former and to dominate JPDA in terms of the latter. Last, we consider an additional DA method that is a modification of PDA by incorporating a weighting scheme based on distances between position estimates and measurements. This distance-weighting approach, when combined with PDA, has been shown to enhance the tracking accuracy of PDA without significant change in the computation burden. Since PDA constitutes a crucial building block of LSPA, we hypothesize that DWPDA, when integrated with LSPA, would perform better under the two performance metrics above. Contrary to expectations, the distance-weighting approach does not enhance the performance of LSPA, whether in terms of tracking accuracy or computation time.

CenterTrack3D: Improved CenterTrack More Suitable for Three-Dimensional Objects

Abstract Compared with two-dimensional (2D) multi-object tracking (MOT) algorithms, three-dimensional (3D) multi-object tracking algorithms have more research significance and broad application prospects in the unmanned vehicles research field. Aiming at the problem of 3D multi-object detection and tracking, in this paper, the multi-object tracker CenterTrack, which focuses on 2D multi-object tracking task while ignoring object 3D information, is improved mainly from two aspects of detection and tracking, and the improved network is called CenterTrack3D. In terms of detection, CenterTrack3D uses the idea of attention mechanism to optimize the way that the previous-frame image and the heatmap of previous-frame tracklets are added to the current-frame image as input, and second convolutional layer of the hm output head is replaced by dynamic convolution layer, which further improves the ability to detect occluded objects. In terms of tracking, a cascaded data association algorithm based on 3D Kalman filter is proposed to make full use of the 3D information of objects in the image and increase the robustness of the 3D multi-object tracker. The experimental results show that, compared with the original CenterTrack and the existing 3D multi-object tracking methods, CenterTrack3D achieves 88.75% MOTA for cars and 59.40% MOTA for pedestrians and is very competitive on the KITTI tracking benchmark test set.

A New Real-Time Detection and Tracking Method in Videos for Small Target Traffic Signs

It is a challenging task for self-driving vehicles in Real-World traffic scenarios to find a trade-off between the real-time performance and the high accuracy of the detection, recognition, and tracking in videos. This issue is addressed in this paper with an improved YOLOv3 (You Only Look Once) and a multi-object tracking algorithm (Deep-Sort). First, data augmentation is employed for small sample traffic signs to address the problem of an extremely unbalanced distribution of different samples in the dataset. Second, a new architecture of YOLOv3 is proposed to make it more suitable for detecting small targets. The detailed method is (1) removing the output feature map corresponding to the 32-times subsampling of the input image in the original YOLOv3 structure to reduce its computational costs and improve its real-time performances; (2) adding an output feature map of 4-times subsampling to improve its detection capability for the small traffic signs; (3) Deep-Sort is integrated into the detection method to improve the precision and robustness of multi-object detection, and the tracking ability in videos. Finally, our method demonstrated better detection capabilities, with respect to state-of-the-art approaches, which precision, recall and mAP is 91%, 90%, and 84.76% respectively.

Research on Pedestrian Multi Object Tracking Based on FairMot Transfer Learning

Object tracking is divided into two processes: multi object detection and multi object tracking. In this paper, the multi object detection is studied firstly, and the multi object detection algorithms based on prior box anchor and anchor-free are analyzed and compared. Then in the second process of multi object tracking, multi object tracking algorithm are researched: the multi object tracking algorithm based on prior box anchor, the multi object tracking algorithm based on anchor-free. Finally, Finally, an experiment is carried out to track the pedestrians on the road by using FairMot transfer learning, and the metrics of MOTA, IDF1, IDS, MT, ML and FPS are analyzed. Experiments show that under the method of using transfer learning, MOTA can be improved by 4.2 percentage points.

Real-Time Multiobject Tracking Based on Multiway Concurrency.

This paper explored a pragmatic approach to research the real-time performance of a multiway concurrent multiobject tracking (MOT) system. At present, most research has focused on the tracking of single-image sequences, but in practical applications, multiway video streams need to be processed in parallel by MOT systems. There have been few studies on the real-time performance of multiway concurrent MOT systems. In this paper, we proposed a new MOT framework to solve multiway concurrency scenario based on a tracking-by-detection (TBD) model. The new framework mainly focuses on concurrency and real-time based on limited computing and storage resources, while considering the algorithm performance. For the former, three aspects were studied: (1) Expanded width and depth of tracking-by-detection model. In terms of width, the MOT system can support the process of multiway video sequence at the same time; in terms of depth, image collectors and bounding box collectors were introduced to support batch processing. (2) Considering the real-time performance and multiway concurrency ability, we proposed one kind of real-time MOT algorithm based on directly driven detection. (3) Optimization of system level—we also utilized the inference optimization features of NVIDIA TensorRT to accelerate the deep neural network (DNN) in the tracking algorithm. To trade off the performance of the algorithm, a negative sample (false detection sample) filter was designed to ensure tracking accuracy. Meanwhile, the factors that affect the system real-time performance and concurrency were studied. The experiment results showed that our method has a good performance in processing multiple concurrent real-time video streams.

Multiple objects tracking in the UAV system based on hierarchical deep high-resolution network

Robust and high-performance visual multi-object tracking is a big challenge in computer vision, especially in a drone scenario. In this paper, an online Multi-Object Tracking (MOT) approach in the UAV system is proposed to handle small target detections and class imbalance challenges, which integrates the merits of deep high-resolution representation network and data association method in a unified framework. Specifically, while applying tracking-by-detection architecture to our tracking framework, a Hierarchical Deep High-resolution network (HDHNet) is proposed, which encourages the model to handle different types and scales of targets, and extract more effective and comprehensive features during online learning. After that, the extracted features are fed into different prediction networks for interesting targets recognition. Besides, an adjustable fusion loss function is proposed by combining focal loss and GIoU loss to solve the problems of class imbalance and hard samples. During the tracking process, these detection results are applied to an improved DeepSORT MOT algorithm in each frame, which is available to make full use of the target appearance features to match one by one on a practical basis. The experimental results on the VisDrone2019 MOT benchmark show that the proposed UAV MOT system achieves the highest accuracy and the best robustness compared with state-of-the-art methods.

SmartSORT: an MLP-based method for tracking multiple objects in real-time

With the recent advances in the object detection research field, tracking-by-detection has become the leading paradigm adopted by multi-object tracking algorithms. By extracting different features from detected objects, those algorithms can estimate the similarities and association patterns of objects along with successive frames. However, since similarity functions applied by tracking algorithms are handcrafted, it is difficult to use them in new contexts. In this study, it is investigated the use of artificial neural networks to learning a similarity function that can be used among detections. During training, multilayer perceptron (MLP) neural networks were introduced to correct and incorrect association patterns, sampled from a pedestrian tracking data set. For such, different motion and appearance feature combinations have been explored. Finally, a trained MLP has been inserted into a multiple-object tracking framework, which has been assessed on the MOT Challenge benchmark. Throughout the experiments, the proposed tracker matched the results obtained by state-of-the-art methods by scoring a tracking accuracy of 60.4%, while running 58% faster than DeepSORT, a recent and similar method used as a baseline. After all, this work demonstrates its method can be automatically trained for different tracking contexts and it has highly competitive cost-effectiveness for online real-time tracking applications.

Graph Convolution Neural Network-Based Data Association for Online Multi-Object Tracking

In this paper, a graph convolutional network (GCN)-based multi-object tracking (MOT) algorithm, consisting of a module for extracting the initial features and a module for updating the features, that estimates the affinity between nodes is proposed. The feature extraction module utilizes the pose feature of the object such that the tracking is correct even when the object is partially occluded. Unlike previous graph neural network (GNN)-based MOT methods, this study is based on a GCN and includes a new feature update mechanism, which is updated by combining the output of the neural network and the node similarity between the tracker and detection nodes for each layer. The node feature is updated by aggregating the updated edge feature and the connection strength between the tracker and detection. In each GCN layer, the three networks for the node, edge update, and edge classification were designed to minimize the network parameters to enable faster MOT compared to other GCN-based MOTs. The entire GCN network was designed to learn end-to-end through an affinity loss. The experimental results for the MOT16 and 17 challenge datasets show that the proposed method achieves a superior or similar performance in terms of tracking accuracy and speed compared to state-of-the-art methods, including GCN-based MOT.

Distributed Multi-Object Tracking Under Limited Field of View Sensors

We consider the challenging problem of tracking multiple objects using a distributed network of sensors. In the practical setting of nodes with limited field of views (FoVs), computing power and communication resources, we develop a novel distributed multi-object tracking algorithm. To accomplish this, we first formalise the concept of label consistency, determine a sufficient condition to achieve it and develop a novel \textit{label consensus approach} that reduces label inconsistency caused by objects' movements from one node's limited FoV to another. Second, we develop a distributed multi-object fusion algorithm that fuses local multi-object state estimates instead of local multi-object densities. This algorithm: i) requires significantly less processing time than multi-object density fusion methods; ii) achieves better tracking accuracy by considering Optimal Sub-Pattern Assignment (OSPA) tracking errors over several scans rather than a single scan; iii) is agnostic to local multi-object tracking techniques, and only requires each node to provide a set of estimated tracks. Thus, it is not necessary to assume that the nodes maintain multi-object densities, and hence the fusion outcomes do not modify local multi-object densities. Numerical experiments demonstrate our proposed solution's real-time computational efficiency and accuracy compared to state-of-the-art solutions in challenging scenarios. We also release source code at https://github.com/AdelaideAuto-IDLab/Distributed-limitedFoV-MOT for our fusion method to foster developments in DMOT algorithms.

An Uncertainty-Aware Performance Measure for Multi-Object Tracking

Evaluating the performance of multi-object tracking (MOT) methods is not straightforward, and existing performance measures fail to consider all the available uncertainty information in the MOT context. This can lead practitioners to select models which produce uncertainty estimates of lower quality, negatively impacting any downstream systems that rely on them. Additionally, most MOT performance measures have hyperparameters, which makes comparisons of different trackers less straightforward. We propose the use of the negative log-likelihood (NLL) of the multi-object posterior given the set of ground-truth objects as a performance measure. This measure takes into account all available uncertainty information in a sound mathematical manner without hyperparameters. We provide efficient algorithms for approximating the computation of the NLL for several common MOT algorithms, show that in some cases it decomposes and approximates the widely-used GOSPA metric, and provide several illustrative examples highlighting the advantages of the NLL in comparison to other MOT performance measures.

A 3D Multiobject Tracking Algorithm of Point Cloud Based on Deep Learning

3D multiobject tracking (MOT) is an important part of road condition detection and hazard warning algorithm in roadside systems and autonomous driving systems. There is a tricky problem in 3D MOT that the identity of occluded object switches after it reappears. Given the good performance of the 2D MOT, this paper proposes a 3D MOT algorithm with deep learning based on the multiobject tracking algorithm. Firstly, a 3D object detector was used to obtain oriented 3D bounding boxes from point clouds. Secondly, a 3D Kalman filter was used for state estimation, and reidentification algorithm was used to match feature similarity. Finally, data association was conducted by combining Hungarian algorithm. Experiments show that the proposed method can still match the original trajectory after the occluded object reappears and run at a rate of 59 FPS, which has achieved advanced results in the existing 3D MOT system.