Multi-Object Tracking Accuracy Research Articles

Online Multi-Object Tracking in Videos Based on Features Detected by YOLO

With the rapid development of different applications that rely on multi-object detection and tracking, significant attention has been brought toward improving the performance of these methods. Recently, Artificial Neural Networks (ANNs) have shown outstanding performance in different applications, where objects detection and tracking are no exception. In this paper, we proposed a new object tracking method based on descriptors extracted using the convolutional filters of the YOLOv3 neural network. As these features are detected and processed during the detection phase, the proposed method has exploited these features to produce efficient and robust descriptors. The proposed method has shown better performance, compared to state-of-the-art methods, by producing better predictions using less computations. The evaluation results show that the proposed method has been able to process an average of 207.6 frames per second to track objects with 67.6% Multi-Object Tracking Accuracy (MOTA) and 89.1% Multi-Object Tracking Precision (MOTP).

An AIoT Monitoring System for Multi-Object Tracking and Alerting

Pig farmers want to have an effective solution for automatically detecting and tracking multiple pigs and alerting their conditions in order to recognize disease risk factors quickly. In this paper, therefore, we propose a novel monitoring system using an Artificial Intelligence of Things (AIoT) technique combining artificial intelligence and Internet of Things (IoT). The proposed system consists of AIoT edge devices and a central monitoring server. First, an AIoT edge device extracts video frame images from a CCTV camera installed in a pig pen by a frame extraction method, detects multiple pigs in the images by a faster region-based convolutional neural network (RCNN) model, and tracks them by an object center-point tracking algorithm (OCTA) based on bounding box regression outputs of the faster RCNN. Finally, it sends multi-pig tracking images to the central monitoring server, which alerts them to pig farmers through a social networking service (SNS) agent in cooperation with an oneM2M-compliant IoT alerting method. Experimental results showed that the multi-pig tracking method achieved the multi-object tracking accuracy performance of about 77%. In addition, we verified alerting operation by confirming the images received in the SNS smartphone application.

Multimodal Multiobject Tracking by Fusing Deep Appearance Features and Motion Information

Multiobject Tracking (MOT) is one of the most important abilities of autonomous driving systems. However, most of the existing MOT methods only use a single sensor, such as a camera, which has the problem of insufficient reliability. In this paper, we propose a novel Multiobject Tracking method by fusing deep appearance features and motion information of objects. In this method, the locations of objects are first determined based on a 2D object detector and a 3D object detector. We use the Nonmaximum Suppression (NMS) algorithm to combine the detection results of the two detectors to ensure the detection accuracy in complex scenes. After that, we use Convolutional Neural Network (CNN) to learn the deep appearance features of objects and employ Kalman Filter to obtain the motion information of objects. Finally, the MOT task is achieved by associating the motion information and deep appearance features. A successful match indicates that the object was tracked successfully. A set of experiments on the KITTI Tracking Benchmark shows that the proposed MOT method can effectively perform the MOT task. The Multiobject Tracking Accuracy (MOTA) is up to 76.40% and the Multiobject Tracking Precision (MOTP) is up to 83.50%.

Multipedestrian Online Tracking Based on Social Force-Predicted Deformable Key-Points Mapping via Compressive Sensing

Multipedestrian Online Tracking and positioning are the research hotspots and difficulties in the field of computer vision. In this article, we propose a multipedestrian online tracking method based on social force-predicted deformable key-points mapping and compressive sensing. First, part affinity fields are utilized to detect body key-points of pedestrians, and then rectangle boundaries of moving pedestrians can be constrained based on these key-points, the outer rectangle of the key-points replacing the conventional bounding box. Since pedestrian tracking is vulnerable to occlusion and dynamic deformation, a deformable convolution network is applied to extract delicate features. There is one more point, based on the theory of compressive sensing, a very sparse measurement matrix is designed to efficiently compress the features for the pedestrians, which maps features from high-dimensional space to low-dimensional space and preserves the structure of the image feature space of pedestrians. Compressed features are used to measure the similarity between pedestrians in adjacent frames. Furthermore, a pedestrian social force model is adopted, which can predict the trajectory of pedestrians to deal with the problems of partial occlusion and similar appearance, meanwhile making the estimated location more accurately. Finally, extensive experiments on the MOT16 benchmark demonstrate the superior performance of the proposed tracking method. The main tracking performance parameters are significantly improved, with multiobject tracking accuracy and multiobject tracking precision reaching 62.8% and 81.2%, respectively.

3D Pedestrian tracking using local structure constraints

Tracking pedestrians based on visual sensors has many diverse applications, among them autonomous driving. Besides obtaining high recall, maintaining the consistency of tracked trajectories during data association is one of the most crucial issues of any tracker. This issue has been tackled in the literature for some time, taking advantage of geometry cues for improving the pairwise matching of detections across consecutive frames. However, this idea has only been employed in a simple way and not thoroughly leveraged in existing studies, i.e., only 2D information is utilized that cannot help to completely understand the real-world geometry in 3D space. Motivated by this observation, in this paper, we present a new method called 3D-TLSR (3D pedestrian tracking using local structure refinement). We use stereo images and expand the idea of geometry cues into 3D space to improve the association of existing trajectories and new detections. We divide the assignment optimization into two steps: (1) determining trajectories whose assignments are strongly believed to be correct, which we call anchors and (2) employing geometry constraints between the anchors and their nearby trajectories in 3D space to improve the matching of less reliable assignments of the first step. In addition, we suggest a simple approach to compute and correct the velocity of a tracked person so that we can better recover missed detections. Experimental results on the well known KITTI tracking benchmark, the ETHMS data set, as well as a self-generated dataset show that our tracker yields comparable results to other state-of-the-art methods with (for KITTI) multi object tracking accuracy (MOTA) of 54.00, which is the best online result among all investigated approaches, multi object tracking precision (MOTP) of 73.03, which is the best of all reported values, and mostly tracked (MT) of 29.55, being the second-best result. On the ETHMS dataset, our approach obtains best results with large margins for recall, precision, and MT, while maintaining a reasonable low number of Id switches (IDs) and fragmentation (FG). These findings confirms the effectiveness of our proposed association method and velocity estimation approach.

Tracking multiple construction workers through deep learning and the gradient based method with re-matching based on multi-object tracking accuracy

Multiple construction worker tracking is an active research area critical to the planning of the job site. Challenges in multiple construction worker tracking include miss detection and mismatch due to occlusion and identity switches. To the best knowledge of the authors, the mismatch is not reported in the literature of construction for image based single camera multiple worker tracking. As a result, the mismatch should be taken into account through a representative performance index such as the Multi-Object Tracking Accuracy (MOTA). This work aims to improve the performance of the current multiple worker tracking through an approach composed of three stages: detection, matching and re-matching. In the detection stage, the deep learning detector, Mask R-CNN, is utilized. In the matching stage, we attempt to track workers between consecutive image frames through a gradient based method with feature based comparison. Several cost means and matching methods have been experimented for model selection. Trajectories of tracking objects are derived in this stage. The best cost measurements and matching methods are recommended. Trajectories of tracking objects could be interrupted because of miss detection or mismatch. We call those broken trajectories, without matched detections, orphans. In the re-matching stage, we attempt to recover unmatched detections in the current frame with previous orphans based on extracted features. A competitive MOTA of 56.7% was obtained from the proposed approach over MOTA of 55.9% from the state-of-the-art Detect-And-Track model on a human tracking benchmark dataset. On construction job sites, we have tested the approach with 4 testing videos, resulting in a total MOTA of 81.8%, average MOTA per video of 79.0% and standard deviation of 13.0%, while the maximum and minimum MOTAs are 96.0% and 69.0%, respectively. As a result, the proposed work could potentially provide better multiple worker tracking on the construction job site. Additionally, to have a better representation of the tracking errors, this work suggests to utilize the MOTA for multiple construction worker tracking.

Automated Individual Pig Localisation, Tracking and Behaviour Metric Extraction Using Deep Learning

Individual pig tracking is key to stepping away from group-level treatment and towards individual pig care. By doing so we can monitor individual pig behaviour changes over time and use these as indicators of health and well-being, which, in turn, will assist in the early detection of disease allowing for earlier and more effective intervention. However, it is a much more computationally challenging than performing this task at group level; mistakes in identification and tracking accumulate and, over time, provide noise measures. We combine a deep CNN object localisation method, Faster Region-based convolutional neural network (R-CNN), with two potential real-time multi-object tracking methods in order to create a complete system that can autonomously localise and track individual pigs allowing for the extraction of metrics pertaining to individual pig behaviours from RGB cameras. We evaluate two different transfer learning strategies to adapt Faster R-CNN to our pig detection dataset that is more challenging than conventional tracking benchmark datasets. We are able to localise pigs in individual frames with 0.901 mean average precision (mAP), which then allows us to track individual pigs across video footage with 92% Multi-Object Tracking Accuracy (MOTA) and 73.4% Identity F1-Score (IDF1), and re-identify them after occlusions and dropped frames with 0.862 mAP (0.788 Rank 1 cumulative matching characteristic (CMC)). From these tracks we extract individual behavioural metrics for total distance travelled, time spent idle, and average speed with less than 0.015 mean squared error (MSE) for each. Changes in all these behavioural metrics have value in the detection of pig health and wellbeing.

Online Multi-Object Tracking With GMPHD Filter and Occlusion Group Management

In this paper, we propose an efficient online multi-object tracking framework based on the GMPHD filter and occlusion group management scheme where the GMPHD filter utilizes hierarchical data association to reduce the false negatives caused by miss detection. The hierarchical data association consists of two steps: detection-to-track and track-to-track associations, which can recover the lost tracks and their switched IDs. In addition, the proposed framework is equipped with an object grouping management scheme which handles occlusion problems with two main parts. The first part is "track merging" which can merge the false positive tracks caused by false positive detections from occlusions, where the false positive tracks are usually occluded with a measure. The measure is the occlusion ratio between visual objects, sum-of-intersection-over-area (SIOA) we defined instead of the IOU metric. The second part is "occlusion group energy minimization (OGEM)" which prevents the occluded true positive tracks from false "track merging". We define each group of the occluded objects as an energy function and find an optimal hypothesis which makes the energy minimal. We evaluate the proposed tracker in benchmark datasets such as MOT15 and MOT17 which are built for multi-person tracking. An ablation study in training dataset shows that not only "track merging" and "OGEM" complement each other but also the proposed tracking method has more robust performance and less sensitive to parameters than baseline methods. Also, SIOA works better than IOU for various sizes of false positives. Experimental results show that the proposed tracker efficiently handles occlusion situations and achieves competitive performance compared to the state-of-the-art methods. Especially, our method shows the best multi-object tracking accuracy among the online and real-time executable methods.

Convolutional Recurrent Predictor: Implicit Representation for Multi-Target Filtering and Tracking

Defining a multi-target motion model, which is an important step of tracking algorithms, can be very challenging. Using fixed models (as in several generative Bayesian algorithms, such as Kalman filters) can fail to accurately predict sophisticated target motions. On the other hand, sequential learning of the motion model (for example, using recurrent neural networks) can be computationally complex and difficult due to the variable unknown number of targets. In this paper, we propose a multi-target filtering and tracking algorithm which learns the motion model, simultaneously for all targets, from an implicitly represented state map and performs spatio-temporal data prediction. To this end, the multi-target state is modelled over a continuous hypothetical target space, using random finite sets and Gaussian mixture probability hypothesis density formulations. The prediction step is recursively performed using a deep convolutional recurrent neural network with a long short-term memory architecture, which is trained as a regression block, on the fly, over "probability density difference" maps. Our approach is evaluated over widely used pedestrian tracking benchmarks, remarkably outperforming state-of-the-art multi-target filtering algorithms, while giving competitive results when compared with other tracking approaches: The proposed approach generates an average 40.40 and 62.29 optimal sub-pattern assignment (OSPA) errors on MOT15 and MOT16/17 datasets, respectively, while producing 62.0%, 70.0% and 66.9% multi-object tracking accuracy (MOTA) on MOT16/17, PNNL Parking Lot and PETS09 pedestrian tracking datasets, respectively, when publicly available detectors are used.