Real-time Tracking Of Multiple Objects Research Articles

Deep learning and multi-modal fusion for real-time multi-object tracking: Algorithms, challenges, datasets, and comparative study

Real-time multi-object tracking (MOT) is a complex task involving detecting and tracking multiple objects. After the objects are detected, they are assigned markers, and their trajectories are tracked in real-time. The scientific community is intrigued by the possibilities of utilizing MOT technology in the context of smart cities. Their primary focus lies in the domains of intelligent transportation, detection of vehicles and pedestrians, crowd surveillance, and public safety. Deep learning techniques have been developed in recent years to effectively tackle the challenges of real-time MOT tasks and enhance tracking performance. Environmental perception within smart traffic applications relies heavily on sensor data fusion. In traffic scenarios, a thoughtful approach involves utilizing a combination of sensors and cameras to detect and track targets while gathering valuable data effectively. However, it faces challenges when it comes to detecting and tracking objects that are in motion, have complex changes in appearance, or are in crowded scenes. This paper explores the foundational standard for real-time Multiple Object Tracking tasks. We prioritize the examination of quantitative measures by conducting a comprehensive analysis of widely utilized benchmark datasets and metrics. This study also investigates established embedding techniques and multi-modal fusion methods within real-time multi-target tracking algorithms. Each strategy will be classified and assessed according to a predefined set of principles. The paper presents a comprehensive analysis and visual representation of various MOT strategies. Finally, this paper aims to present an overview of the current challenges faced by the MOT mission, as well as the potential objectives that lie ahead.

Deep Neural Network-Based Dynamical Object Recognition and Robust Multiobject Tracking Technique for Onboard Unmanned Aerial Vehicle’s Computer Vision-Based Systems

Computer vision-based systems seem highly perspective for semantic analysis of the dynamical objects. However, considering dynamical object recognition and tracking from the unmanned aerial vehicle the task to design a robust model for data association is highly challenging due to additional issues, e.g., image degradation, non-fixed object camera distance and shooting focus, and real-time issues. Thus, we propose an accurate deep neural network-based dynamical object recognition and robust multi-object tracking technique based on bidirectional LSTM with the optimized motion and appearance gates as a multi-object tracking backbone, supported by an advanced single-shot detector network improved with residual prediction model and implemented DenseNet network as well as YOLOv4eff network as feature extraction. The technique has been trained on VisDrone 2022 and UAVDT datasets with the side-shoot dynamical objects at a height of up to 50 meters. The performance analysis on the test stage performed on 7 metrics demonstrate that the proposed technique surpasses by accuracy and robustness ability of other state-of-the-art techniques based on 2 cumulative MOTA and MOTP, as well as MT and IDsw. In particular, we have dramatically decreased the number of IDsw which implies a better capability to handle several occlusions, which is a desirable property in real-time multiple object tracking. We have pointed out the sensitivity of the tracking performance of our technique on the number of utilizing different sequence lengths and have defined an optimum. Finally, the applicability and reliability of the proposed technique for onboard UAV computer-based systems have been discussed.

Enhancing Detection Quality Rate with a Combined HOG and CNN for Real-Time Multiple Object Tracking across Non-Overlapping Multiple Cameras.

Multi-object tracking in video surveillance is subjected to illumination variation, blurring, motion, and similarity variations during the identification process in real-world practice. The previously proposed applications have difficulties in learning the appearances and differentiating the objects from sundry detections. They mostly rely heavily on local features and tend to lose vital global structured features such as contour features. This contributes to their inability to accurately detect, classify or distinguish the fooling images. In this paper, we propose a paradigm aimed at eliminating these tracking difficulties by enhancing the detection quality rate through the combination of a convolutional neural network (CNN) and a histogram of oriented gradient (HOG) descriptor. We trained the algorithm with an input of 120 × 32 images size and cleaned and converted them into binary for reducing the numbers of false positives. In testing, we eliminated the background on frames size and applied morphological operations and Laplacian of Gaussian model (LOG) mixture after blobs. The images further underwent feature extraction and computation with the HOG descriptor to simplify the structural information of the objects in the captured video images. We stored the appearance features in an array and passed them into the network (CNN) for further processing. We have applied and evaluated our algorithm for real-time multiple object tracking on various city streets using EPFL multi-camera pedestrian datasets. The experimental results illustrate that our proposed technique improves the detection rate and data associations. Our algorithm outperformed the online state-of-the-art approach by recording the highest in precisions and specificity rates.

FAANet: feature-aligned attention network for real-time multiple object tracking in UAV videos

FAANet: feature-aligned attention network for real-time multiple object tracking in UAV videos

Real-Time Multiple Object Visual Tracking for Embedded GPU Systems

Real-time visual object tracking provides every object of interest with a unique identity and a trajectory across video frames. This is a fundamental task of many video analytics applications, such as traffic monitoring or video surveillance in general. The development of real-time multiple object tracking systems on low-power edge devices as IoT nodes, without compromising accuracy, is a challenge due to the limited computing capacity of said devices. This might rule out the best in-class computer vision solutions, which, nowadays, are based on deep learning, and thus, they are very hardware demanding. This article meets this challenge with a multiple object detection and tracking system that employs cutting-edge deep learning architectures on an embedded GPU while operating in real time. For this purpose, a system has been designed that extends a joint architecture of tracking and detection by adding a module comprised of appearance-based and movement-based trackers that allow to maintain the identity of the objects of interest for longer periods of time while alleviating the burden of the detector. Our system is mapped onto an embedded GPU platform, cutting down power consumption significantly with respect to a server GPU. Tracking performance metrics show a 51.1% in multiple object tracking accuracy (MOTA) on the MOT16 data set. This, in conjunction with a real-time processing speed of 25.2 FPS for up to 45 simultaneous objects and low-power consumption of 15 W, make our system an ideal solution for a wide range of video analytics applications.

Analysis Based on Recent Deep Learning Approaches Applied in Real-Time Multi-Object Tracking: A Review

The deep learning technique has proven to be effective in the classification and localization of objects on the image or ground plane over time. The strength of the technique’s features has enabled researchers to analyze object trajectories across multiple cameras for online multi-object tracking (MOT) systems. In the past five years, these technical features have gained a reputation in handling several real-time multiple object tracking challenges. This contributed to the increasing number of proposed deep learning methods (DLMs) and networks seen by the computer vision community. The technique efficiently handled various challenges in real-time MOT systems and improved overall tracking performance. However, it experienced difficulties in the detection and tracking of objects in overcrowded scenes and motion variations and confused appearance variations. Therefore, in this paper, we summarize and analyze the 95 contributions made in the past five years on deep learning-based online MOT methods and networks that rank highest in the public benchmark. We review their expedition, performance, advantages, and challenges under different experimental setups and tracking conditions. We also further categorize these methods and networks into four main themes: Online MOT Based Detection Quality and Associations, Real-Time MOT with High-Speed Tracking and Low Computational Costs, Modeling Target Uncertainty in Online MOT, and Deep Convolutional Neural Network (DCNN), Affinity and Data Association. Finally, we discuss the ongoing challenges and directions for future research.

Towards real-time multiple surgical tool tracking

ABSTRACT Surgical tool tracking is an essential building block for computer-assisted interventions (CAI) and applications like video summarisation, workflow analysis and surgical navigation. Vision-based instrument tracking in laparoscopic surgical data faces significant challenges such as fast instrument motion, multiple simultaneous instruments and re-initialisation due to out-of-view conditions or instrument occlusions. In this paper, we propose a real-time multiple object tracking framework for whole laparoscopic tools, which extends an existing single object tracker. We introduce a geometric object descriptor, which helps with overlapping bounding box disambiguation, fast motion and optimal assignment between existing trajectories and new hypotheses. We achieve 99.51% and 75.64% average accuracy on ex-vivo robotic data and in-vivo laparoscopic sequences respectively from the Endovis’15 Instrument Tracking Dataset. The proposed geometric descriptor increased the performance on laparoscopic data by 32%, significantly reducing identity switches, false negatives and false positives. Overall, the proposed pipeline can successfully recover trajectories over long-sequences and it runs in real-time at approximately 25–29 fps.

Real-Time Tracking of Single and Multiple Objects from Depth-Colour Imagery Using 3D Signed Distance Functions

We describe a novel probabilistic framework for real-time tracking of multiple objects from combined depth-colour imagery. Object shape is represented implicitly using 3D signed distance functions. Probabilistic generative models based on these functions are developed to account for the observed RGB-D imagery, and tracking is posed as a maximum a posteriori problem. We present first a method suited to tracking a single rigid 3D object, and then generalise this to multiple objects by combining distance functions into a shape union in the frame of the camera. This second model accounts for similarity and proximity between objects, and leads to robust real-time tracking without recourse to bolt-on or ad-hoc collision detection.

Real-time tracking of multiple objects using adaptive correlation filters with complex constraints

A real-time system for classification and tracking of multiple moving objects is proposed. The system employs a bank of composite correlation filters with complex constraints implemented in parallel on a graphics processing unit. When a scene frame is captured, the system splits the frame into several fragments on the base of a modeling kinematic prediction of target's locations. The fragments are processed with a bank of adaptive filters. The filters are synthesized with the help of an iterative algorithm, which optimizes discrimination capability for each target. Using complex constraints in the filter design, multiple objects in the input frame can be detected and classified by analyzing the intensity and phase distributions on the output complex correlation plane for each fragment. The performance of the proposed system in terms of tracking accuracy, classification efficiency and time expenses is tested and discussed with synthetic and real input-scene sequences. The results are compared with those of common techniques based on correlation filtering.

Resolving the ambiguity of real-time multiple object tracking using static cameras

Recently, because of its importance in computer vision and surveillance systems, object tracking has progressed rapidly. In this paper a novel strategy for tracking multiple objects using static cameras is introduced, which can be used to grant a cheap, easy installation, and robust tracking system. The proposed tracking strategy is based on scenes captured by static video cameras. Each camera is attached to a workstation that analyses its stream. All workstations are connected to a tracking server, which harmonises the system, collects the data, and creates the output spatial-tempo database. Our contribution comes in two issues. The first is to present a new methodology for transforming the image coordinates of an object to its real coordinates. The second is to offer a flexible event-based object tracking strategy, which has been tested over a CAD of soccer game environment. Preliminary experimental results show the robust performance of the proposed tracking strategy.

A Greedy Searching Algorithm for Multiple Object Tracking and Occlusion Handling

This paper presents a novel real-time multiple object tracking algorithm,which contains three parts:region correlation based foreground segmentation,merging-splitting based data association and greedy searching based occluded object localization.The main characteristics of the proposed algorithm are summarized as follows:1) the multiple object tracking and occlusion handling problem is successfully changed into an image classification problem with prior knowledge of object number and feature;2) a highly effcient greedy searching method is presented to meet real-time capability;3) it has good performance in expansibility,and it has no constraints about the number of occluded objects,the occlusion ratio and the object s motion model.Experiment results with hand labeled IBM database demonstrate that the method is effective and effcient.

Real-time tracking of multiple objects in space-variant vision based on magnocellular visual pathway

In this paper, we propose a space-variant image representation model based on properties of magnocellular visual pathway, which perform motion analysis, in human retina. Then, we present an algorithm for the tracking of multiple objects in the proposed space-variant model. The proposed space-variant model has two effective image representations for object recognition and motion analysis, respectively. Each image representation is based on properties of two types of ganglion cell, which are the beginning of two basic visual pathways; one is parvocellular and the other is magnocellular. Through this model, we can get the efficient data reduction capability with no great loss of important information. And, the proposed multiple objects tracking method is restricted in space-variant image. Typically, an object-tracking algorithm consists of several processes such as detection, prediction, matching, and updating. In particular, the matching process plays an important role in multiple objects tracking. In traditional vision, the matching process is simple when the target objects are rigid. In space-variant vision, however, it is very complicated although the target is rigid, because there may be deformation of an object region in the space-variant coordinate system when the target moves to another position. Therefore, we propose a deformation formula in order to solve the matching problem in space-variant vision. By solving this problem, we can efficiently implement multiple objects tracking in space-variant vision.