Object Tracking Method Research Articles

DASFTOT: Dual attention spatiotemporal fused transformer for object tracking

The Siamese network method is widely applied in the field of object tracking. The transformer-based tracker achieves state-of-the-art tracking results. However, these methods cannot effectively fuse local and global features of video images and cannot pay more attention to tracking objects spatiotemporally. In this paper, we proposed a new object tracking method (DASFTOT), which includes a backbone network, transformer mechanism and bounding prediction box. First, we use a 3D CNN to extract motion information. Second, we superimpose important temporal and spatial information through a dual attention spatiotemporal fused transformer (DASFT) to fuse local and global spatiotemporal features and calculate the correlation between templates and search regions. Third, to improve the robustness of tracking, we dynamically update part of the template frame. Finally, we position the tracking object through a bounding prediction box. To verify the effectiveness of the proposed tracker (DASFTOT), experiments on the GOT-10K, LaSOT, TrackingNet, VOT2020 and OTB100 benchmark datasets demonstrated that the proposed tracker was highly comparable to other state-of-the-art methods.

Drone Tracking with Drone using Deep Learning

With the development of technology, studies in fields such as artificial intelligence, computer vision and deep learning are increasing day by day. In line with these developments, object tracking and object detection studies have spread over wide areas. In this article, a study is presented by simulating two different drones, a leader and a follower drone, accompanied by deep learning algorithms. Within the scope of this study, it is aimed to perform a drone tracking with drone in an autonomous way. Two different approaches are developed and tested in the simulator environment within the scope of drone tracking. The first of these approaches is to enable the leader drone to detect the target drone by using object-tracking algorithms. YOLOv5 deep learning algorithm is preferred for object detection. A data set of approximately 2500 images was created for training the YOLOv5 algorithm. The Yolov5 object detection algorithm, which was trained with the created data set, reached a success rate of approximately 93% as a result of the training. As the second approach, the object-tracking algorithm we developed is used. Trainings were carried out in the simulator created in the Matlab environment. The results are presented in detail in the following sections. In this article, some artificial neural networks and some object tracking methods used in the literature are explained.

Ensemble Learning-Based Multi-Cues Fusion Object Tracking in Complex Surveillance Environment.

The vast majority of currently available kernelized correlation filter (KCF)-based trackers simply make use of a single object feature to define the object of interest. It is impossible to avoid tracking instability while working with a wide variety of complex videos. In this piece of research, an ensemble learning-based multi-cues fusion object tracking method is offered as a potential solution to the issue at hand. Using ensemble learning to train multiple kernelized correlation filters with different features in order to obtain the optimal tracking parameters is the primary concept behind the improved KCF-based tracking algorithm. After that, the peak side lobe ratio and the response consistency of two adjacent frames are used to obtain the fusion weight. In addition, an adaptive weighted fusion technique is applied in order to combine the response findings in order to finish the location estimation; finally, the tracking confidence is applied in order to update the tracking model in order to prevent model deterioration. In order to increase the adaptability of the revised algorithm to size-change, a Bayesian estimate model based on scale pyramid has been presented. This model is able to determine the optimal scale of the object, which is the goal of this endeavor. The tracking results of a number of different benchmark movies demonstrate that the algorithm that we have suggested is able to effectively eliminate the effects of interference elements, and that its overall performance is superior to that of the comparison algorithms.

Multiple Object Tracking of Drone Videos by a Temporal-Association Network with Separated-Tasks Structure

The task of multi-object tracking via deep learning methods for UAV videos has become an important research direction. However, with some current multiple object tracking methods, the relationship between object detection and tracking is not well handled, and decisions on how to make good use of temporal information can affect tracking performance as well. To improve the performance of multi-object tracking, this paper proposes an improved multiple object tracking model based on FairMOT. The proposed model contains a structure to separate the detection and ReID heads to decrease the influence between every function head. Additionally, we develop a temporal embedding structure to strengthen the representational ability of the model. By combing the temporal-association structure and separating different function heads, the model’s performance in object detection and tracking tasks is improved, which has been verified on the VisDrone2019 dataset. Compared with the original method, the proposed model improves MOTA by 4.9% and MOTP by 1.2% and has better tracking performance than the models such as SORT and HDHNet on the UAV video dataset.

Multiple object tracking in farmland based on fusion point cloud data

Object detection and tracking in a farmland environment is greatly important to unmanned agricultural machinery navigation technology. Based on the existing methods of fusion point cloud acquisition and object detection, the multiple object tracking method was further researched. Constant velocity model was selected as the motion model, and state estimation was achieved using Kalman filter. Data association between the detectors and trackers was realized through an improved GNN method based on the cost of category conversion. A tracking state machine and two threshold mechanisms were proposed to avoid “combined explosion” disaster. Fusion point cloud data were used for motion judgment to achieve the real motion attributes and speed of the objects based on ground reference. As the results, the average static drift of the tracking method was less than 6.5 cm, the discrimination accuracy of static objects exceeded 75%. Besides, the multiple object tracking precision was less than 35 cm, and the multiple object tracking accuracy exceeded 85%. Therefore, the proposed multiple object tracking method could satisfy the requirements in the farmland environment.

Deep learning based object tracking for 3D microstructure reconstruction.

In medical and material science, 3D reconstruction is of great importance for quantitative analysis of microstructures. After the image segmentation process of serial slices, in order to reconstruct each local structure in volume data, it needs to use precise object tracking algorithm to recognize the same object region in adjacent slice. Suffering from weak representative hand-crafted features, traditional object tracking methods always draw out under-segmentation results. In this work, we have proposed an adjacent similarity based deep learning tracking method (ASDLTrack) to reconstruct 3D microstructure. By transferring object tracking problem to classification problem, it can utilize powerful representative ability of convolutional neural network in pattern recognition. Experiments in three datasets with three metrics demonstrate that our algorithm achieves the promising performance compared to traditional methods.

A motional but temporally consistent physical video examples

Adversarial examples (AEs) attract extensive attention due to their inherent security-related properties of attacking Deep Neural Networks (DNNs) through carefully constructed modifications. Recently, they have been extended to video tasks. Human action recognition based on DNNs is a crucial task in video tasks. AEs of human action recognition models attract much focus and previous works demonstrated the vulnerability of AEs of human action recognition in the digital world. However, the adversarial videos for attacking human action recognition models in the physical world are still in the open stage. The design of physical adversarial videos is crucial for helping to evaluate the robustness of some critical applications based on human action recognition, e.g., surveillance and pedestrian detection. Unlike the digital attacks on action recognition models, the perturbations of physical adversarial videos should be motional but temporally consistent across each whole video. The attacks need to destroy the spatial interactions and temporal interactions in videos. Previously developed attacks for video models in the digital world are difficult to transfer to the physical world. In this paper, we close this gap, and we are the first to attack human action recognition models in the physical world. We first generate a dynamic mask via an improved object tracking method and then use the center location to construct a motion location map. Finally, gradient sharing method is used to generate temporally consistent perturbations and optimize the perturbations into robust patches. Experiments show that these patches can successfully attack a real-time human action recognition system, and the proposed approach has a 77.5% success rate in this setting.

Visual Object Tracking Based on Deep Neural Network

Computer vision systems cannot function without visual target tracking. Intelligent video monitoring, medical treatment, human-computer interaction, and traffic management all stand to benefit greatly from this technology. Although many new algorithms and methods emerge every year, the reality is complex. Targets are often disturbed by factors such as occlusion, illumination changes, deformation, and rapid motion. Solving these problems has also become the main task of visual target tracking researchers. As with the development for deep neural networks and attention mechanisms, object-tracking methods with deep learning show great research potential. This paper analyzes the abovementioned difficult factors, uses the tracking framework based on deep learning, and combines the attention mechanism model to accurately model the target, aiming to improve tracking algorithm. In this work, twin network tracking strategy with dual self-attention is designed. A dual self-attention mechanism is used to enhance feature representation of the target from the standpoint of space and channel, with the goal of addressing target deformation and other problems. In addition, adaptive weights and residual connections are used to enable adaptive attention feature selection. A Siamese tracking network is used in conjunction with the proposed dual self-attention technique. Massive experimental results show our proposed method improves tracking performance, and tracking strategy achieves an excellent tracking effect.

Adaptive Fusion CNN Features for RGBT Object Tracking

Thermal sensors play an important role in intelligent transportation system. This paper studies the problem of RGB and thermal (RGBT) tracking in challenging situations by leveraging multimodal data. A RGBT object tracking method is proposed in correlation filter tracking framework based on short term historical information. Given the initial object bounding box, hierarchical convolutional neural network (CNN) is employed to extract features. The target is tracked for RGB and thermal modalities separately. Then the backward tracking is implemented in the two modalities. The difference between each pair is computed, which is an indicator of the tracking quality in each modality. Considering the temporal continuity of sequence frames, we also incorporate the history data into the weights computation to achieve a robust fusion of different source data. Experiments on three RGBT datasets show the proposed method achieves comparable results to state-of-the-art methods.

Visual Object Tracking Based on Adaptive Background-Awareness and Spatial Constraint

The correlation filter method is effective in visual tracking tasks, whereas it suffers from the boundary effect and filter degradation in complex situations, which can result in suboptimal performance. Aiming at the solving above problem, this study proposes an object tracking method with a discriminant correlation filter, which combines an adaptive background perception and a spatial dynamic constraint. In this method, an adaptive background-awareness strategy is used to filter the background information trained by the interference filter to improve the discriminability between the object and the background. In addition, the spatial regularization term is introduced, and the dynamic change of the real filter and the predefined spatial constraint template is used to optimize filter learning to enhance the spatial information capture ability of the filter model. Experiments on the OTB100, VOT2018, and TrackingNet standard datasets demonstrate that our method achieves favorable accuracy and success rates. Compared with the current popular correlation filter methods, the proposed method can still maintain stable tracking performance with a scene scale variation, complex background, motion blur, and fast motion.

Multiple Traffic Target Tracking with Spatial-Temporal Affinity Network.

Traffic target tracking is a core task in intelligent transportation system because it is useful for scene understanding and vehicle autonomous driving. Most state-of-the-art (SOTA) multiple object tracking (MOT) methods adopt a two-step procedure: object detection followed by data association. The object detection has made great progress with the development of deep learning. However, the data association still heavily depends on hand crafted constraints, such as appearance, shape, and motion, which need to be elaborately trained for a special object. In this study, a spatial-temporal encoder-decoder affinity network is proposed for multiple traffic targets tracking, aiming to utilize the power of deep learning to learn a robust spatial-temporal affinity feature of the detections and tracklets for data association. The proposed spatial-temporal affinity network contains a two-stage transformer encoder module to encode the features of the detections and the tracked targets at the image level and the tracklet level, aiming to capture the spatial correlation and temporal history information. Then, a spatial transformer decoder module is designed to compute the association affinity, where the results from the two-stage transformer encoder module are fed back to fully capture and encode the spatial and temporal information from the detections and the tracklets of the tracked targets. Thus, efficient affinity computation can be applied to perform data association in online tracking. To validate the effectiveness of the proposed method, three popular multiple traffic target tracking datasets, KITTI, UA-DETRAC, and VisDrone, are used for evaluation. On the KITTI dataset, the proposed method is compared with 15 SOTA methods and achieves 86.9% multiple object tracking accuracy (MOTA) and 85.71% multiple object tracking precision (MOTP). On the UA-DETRAC dataset, 12 SOTA methods are used to compare with the proposed method, and the proposed method achieves 20.82% MOTA and 35.65% MOTP, respectively. On the VisDrone dataset, the proposed method is compared with 10 SOTA trackers and achieves 40.5% MOTA and 74.1% MOTP, respectively. All those experimental results show that the proposed method is competitive to the state-of-the-art methods by obtaining superior tracking performance.

Detection and tracking of moving cloud services from video using saliency map model

In cloud computing, the services are observed in the video stream and clustering their pixels is the initial task in service detection. Tracking is the practice to observe or tracking the moments of a given item in each frame. Numerous false positives are included in the frame. Using the Saliency Map Model and the Extended Kalman Filter, the proposed approach can recognize and track moving objects in video. The item is tracked using an Extended Kalman Filter. In the proposed research the evaluation is based on the delay and accuracy of the evaluation parameter. Finally, the suggested method is compared to existing object tracking methods, with an accuracy of greater than 90% attained.

MULTIPLE OBJECT TRACKING USING A TRANSFORM SPACE

Abstract. This paper presents a method for multiple object tracking (MOT) in video streams. The method incorporates the prediction of physical locations of people into a tracking-by-detection paradigm. We predict the trajectories of people on an estimated ground plane and apply a learning-based network to extract the appearance features across frames. The method transforms the detected object locations from image space to an estimated ground space to refine the tracking trajectories. This transform space allows the objects detected from multi-view images to be associated under one coordinate system. Besides, the occluded pedestrians in image space can be well separated in a rectified ground plane where the motion models of the pedestrians are estimated. The effectiveness of this method is evaluated on different datasets by extensive comparisons with state-of-the-art techniques. Experimental results show that the proposed method improves MOT tasks in terms of the number of identity switches (IDSW) and the fragmentations (Frag).

A Siamese Network for real-time object tracking on CPU

<h2>Abstract</h2> Visual object tracking methods depend upon deep networks that can hardly meet real-time processing requirements on mobile platforms with limited computing resources. In this work, we propose a real-time object tracking framework by enhancing a lightweight feature pyramid network with Transformer architecture to construct a robust target-specific appearance model efficiently. We further introduce the pooling attention module to avoid the computation and memory intensity while fusing pyramid features with the Transformer. The optimized tracker operates over 45 Hz on a single CPU, allowing researchers to deploy it on any mobile device with limited power resources.

Biogeography based optimization method for robust visual object tracking

Moving object tracking is one of the applied fields in artificial intelligence and robotic. The main objective of object tracking is to detect and locate targets in video frames of real scenes. Although various methods have been proposed for object tracking so far, tracking in challenging conditions remains an open issue. Recently, different evolutionary and heuristics algorithms like swarm intelligence have been used to address the tracking challenges, which have shown promising performance. In this paper, a new approach based on modified biogeography based optimization (mBBO) method is introduced. The BBO algorithm includes migration and mutation steps. In the migration phase, the search space is properly explored by sharing information between habitats and weaker solutions to improve their position. On the other hand, the mutation phase leads to diversity and change in solutions. In this algorithm, the elitist method has been also used to keep better solutions. The performance of modified BBO tracker has been evaluated on benchmark video datasets and compared with several other tracking methods. Experimental results demonstrate that the proposed method estimates the location of targets with high accuracy and achieves better performance and robustness compared to other trackers.

On the Real Time Object Detection and Tracking

Object detection and tracking is widely used for detecting motions of objects present in images and video.Since last so many decades, numerous real time object detection and tracking methods have been proposed byresearchers. The proposed methods for objects to be tracked till date require some preceding informationassociated with moving objects. In real time object detection and tracking approach segmentation is the initialtask followed by background modeling for the extraction of predefined information including shape of the objects,position in the starting frame, texture, geometry and so on for further processing of the cluster pixels and videosequence of these objects. The object detection and tracking can be applied in the fields like computerized videosurveillance, traffic monitoring, robotic vision, gesture identification, human-computer interaction, militarysurveillance system, vehicle navigation, medical imaging, biomedical image analysis and many more. In thispaper we focus detailed technical review of different methods proposed for detection and tracking of objects. Thecomparison of various techniques of detection and tracking is the purpose of this work.

Data-driven online traffic reconstructions: Interactively optimizing in virtual reality

It is important to realize three-dimensional (3D) online traffic reconstructions based on the video data. Existing methods always use structured data to generate 3D traffic reconstructions and do not process unstructured pixel data. In this paper, we propose a 3D online traffic reconstruction method driven by video data and virtual traffic simulations. There are mainly two steps in our method. The first step is data preprocessing. We extract vehicles’ trajectories from video data and transform them into the geodetic coordinate system. The transformation parameters are optimally solved in multiple human–computer interactions. The second step is trajectory reconstructions. We use the traffic simulation method to predict vehicles’ trajectories and use the Kalman filter to optimize vehicles’ trajectories according to the extracted trajectories and the predicted trajectories. We also optimize the traffic simulation parameters iteratively according to the optimized trajectories when needed. The experimental results show that our method can provide realistic and smooth traffic reconstructions. It has nearly the same efficiency as that of existing online multiple object tracking methods.

EmbeddedPigCount: Pig Counting with Video Object Detection and Tracking on an Embedded Board.

Knowing the number of pigs on a large-scale pig farm is an important issue for efficient farm management. However, counting the number of pigs accurately is difficult for humans because pigs do not obediently stop or slow down for counting. In this study, we propose a camera-based automatic method to count the number of pigs passing through a counting zone. That is, using a camera in a hallway, our deep-learning-based video object detection and tracking method analyzes video streams and counts the number of pigs passing through the counting zone. Furthermore, to execute the counting method in real time on a low-cost embedded board, we consider the tradeoff between accuracy and execution time, which has not yet been reported for pig counting. Our experimental results on an NVIDIA Jetson Nano embedded board show that this “light-weight” method is effective for counting the passing-through pigs, in terms of both accuracy (i.e., 99.44%) and execution time (i.e., real-time execution), even when some pigs pass through the counting zone back and forth.

Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey

Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenges, statistics of existing methods, and provide solutions from the perspectives of DL-based models in three research topics: object detection from the image, object detection from the video, and object tracking from the video. Open datasets related to UAV-dominated object detection and tracking are exhausted, and four benchmark datasets are employed for performance evaluation using some state-of-the-art methods. Finally, prospects and considerations for the future work are discussed and summarized. It is expected that this survey can facilitate those researchers who come from remote sensing field with an overview of DL-based UAV object detection and tracking methods, along with some thoughts on their further developments.

Moving Object Detection and Tracking by Event Frame from Neuromorphic Vision Sensors

Fast movement of objects and illumination changes may lead to a negative effect on camera images for object detection and tracking. Event cameras are neuromorphic vision sensors that capture the vitality of a scene, mitigating data redundancy and latency. This paper proposes a new solution to moving object detection and tracking using an event frame from bio-inspired event cameras. First, an object detection method is designed using a combined event frame and a standard frame in which the detection is performed according to probability and color, respectively. Then, a detection-based object tracking method is proposed using an event frame and an improved kernel correlation filter to reduce missed detection. Further, a distance measurement method is developed using event frame-based tracking and similar triangle theory to enhance the estimation of distance between the object and camera. Experiment results demonstrate the effectiveness of the proposed methods for moving object detection and tracking.