Problem Of Object Tracking Research Articles

Visual tracking with multilayer filter fusion network

We propose the multilayer filter fusion network (MFFN) to address the problem of visual object tracking. In MFFN, the convolutional neural network (CNN) is used to extract the multilayer spatial features and then the convolutional long short-term memory (LSTM) to extract the temporal features of images. The object image centered at the target is cropped and fed into MFFN to obtain the correlation filter and the feature map to discriminate the target from background. The correlation filter is convolved with the corresponding feature map for the same layer to produce the probability map, which is then used to estimate the target position by searching its maximum value. The correlation filter corresponds to the tracked object image that is fed into MFFN and thus contains the appearance changes of target. In our multilayer filter fusion tracking (MFFT) framework, we use two MFFNs with different inputs to track the target via coarse-to-fine location approach. The first one is used to estimate the target position from the entire image and the second one to locate the target from the estimated target position. After the networks are trained off-line they do not require online learning during tracking. Experimental results on the CVPR2013 benchmark demonstrate that our tracking algorithm achieves competitive results compared with other tracking methods.

Variational inference based distributed noise adaptive Bayesian filter

In this paper, we investigate noise covariance adaptive distributed Bayesian filter based on variational Bayesian inference method. In Bayesian filter framework, the joint distribution of state and noise covariance is approximated by variational distributions, where the unknown noise covariance is modeled by inverse-Wishart distribution. With communicating with neighbors, we show that the joint posterior distribution of state and noises can be approximated by recursively performing variational Bayesian expectation (VB-E) and variational Bayesian maximization (VB-M) steps. Then we use the cubature Kalman filter (CKF) to approximate Gaussian interval, and propose a variational Bayesian based distributed adaptive cubature information filter (VB-DACIF). Finally, we illustrate the effectiveness of the proposed estimation algorithm by a cooperative object tracking problem.

SIAMESE NETWORK COMBINED WITH ATTENTION MECHANISM FOR OBJECT TRACKING

Abstract. After the development of deep learning object tracking methods in recent years, the fully convolutional siamese network object tracking algorithm SiamFC has become a more classic deep learning object tracking algorithm. In view of the problem that the accuracy of the tracking results of SiamFC will be reduced in the case of complex backgrounds, this paper introduces the attention mechanism based on the SiamFC, which performs channel and spatial weighting on the feature maps obtained by convolution of the input image. At the same time, the backbone network model of CNN in the algorithm is adjusted, then the siamese network combined with attention mechanism for object tracking is proposed. It can strengthen the effectiveness of the results of feature extraction and enhance the ability of the network model to discriminate targets. In this paper, the algorithm is tested on the OTB2015, VOT2016 and VOT2017 datasets, and compared with multiple object tracking algorithms. Experimental results show that the algorithm in this paper can better solve the complex background problem in object tracking, and has certain advantages compared with other algorithms.

Re-identification framework for long term visual object tracking based on object detection and classification

In this paper, we address the problem of long-term visual object tracking and we present an efficient real-time single object tracking system suitable for integration in autonomous platforms that need to encompass intelligent capabilities. We propose a novel long-term tracking framework for classification based re-detection and tracking, that incorporates state estimation, object re-identification and automated management of tracking and detection results. Our method integrates a novel object re-identification technique which efficiently filters a number of detection candidates and systematically corrects the tracking results. Through extensive experimental validation on the UAV123, UAV20L and TLP datasets, we demonstrate the effectiveness of the proposed system and its advantage over several state-of-the art trackers. The results furthermore highlight the proposed tracker’s ability to handle challenges arising from real-world and long-term scenarios, such as variations in pose, scale, occlusions and out-of-view situations. Furthermore, we propose a variant that is suitable for deployment on autonomous robots, such as Unmanned Aerial Vehicles.

Semantic-Aware Real-Time Correlation Tracking Framework for UAV Videos

Discriminative correlation filter (DCF) has contributed tremendously to address the problem of object tracking benefitting from its high computational efficiency. However, it has suffered from performance degradation in unmanned aerial vehicle (UAV) tracking. This article presents a novel semantic-aware real-time correlation tracking framework (SARCT) for UAV videos to enhance the performance of DCF trackers without incurring excessive computing cost. Specifically, SARCT first constructs an additional detection module to generate ROI proposals and to filter any response regarding the target irrelevant area. Then, a novel semantic segmentation module based on semantic template generation and semantic coefficient prediction is further introduced to capture semantic information, which can provide precise ROI mask, thereby effectively suppressing background interference in the ROI proposals. By sharing features and specific network layers for object detection and semantic segmentation, SARCT reduces parameter redundancy to attain sufficient speed for real-time applications. Systematic experiments are conducted on three typical aerial datasets in order to evaluate the performance of the proposed SARCT. The results demonstrate that SARCT is able to improve the accuracy of conventional DCF-based trackers significantly, outperforming state-of-the-art deep trackers.

Video Object Segmentation and Tracking

Object segmentation and object tracking are fundamental research areas in the computer vision community. These two topics are difficult to handle some common challenges, such as occlusion, deformation, motion blur, scale variation, and more. The former contains heterogeneous object, interacting object, edge ambiguity, and shape complexity; the latter suffers from difficulties in handling fast motion, out-of-view, and real-time processing. Combining the two problems of Video Object Segmentation and Tracking (VOST) can overcome their respective difficulties and improve their performance. VOST can be widely applied to many practical applications such as video summarization, high definition video compression, human computer interaction, and autonomous vehicles. This survey aims to provide a comprehensive review of the state-of-the-art VOST methods, classify these methods into different categories, and identify new trends. First, we broadly categorize VOST methods into Video Object Segmentation (VOS) and Segmentation-based Object Tracking (SOT). Each category is further classified into various types based on the segmentation and tracking mechanism. Moreover, we present some representative VOS and SOT methods of each time node. Second, we provide a detailed discussion and overview of the technical characteristics of the different methods. Third, we summarize the characteristics of the related video dataset and provide a variety of evaluation metrics. Finally, we point out a set of interesting future works and draw our own conclusions.

A survey on online learning for visual tracking

Visual object tracking has become one of the most active research topics in computer vision, which has been growing in commercial development as well as academic research. Many visual trackers have been proposed in the last two decades. Recent studies of computer vision for dynamic scenes include motion detection, object classification, environment modeling, tracking of moving objects, understanding of object behaviors, object identification, and data fusion from multiple sensors. This paper provides an in-depth overview of recent object tracking research. Object tracking tasks in realistic scenario often face challenging problems such as camera motion, occlusion, illumination effect, clutter, and similar appearance. A variety of tracker techniques have been published, which combine multiple techniques to solve multiple visual tracking sub-problems. This paper also reviews the latest research trend in object tracking based on convolutional neural networks, which is receiving growing attention. Finally, the paper discusses the future challenges and research directions for the object tracking problems that still need extensive studies in coming years.

Multiple player tracking in basketball court videos

To build a smart basketball court, one basic task is to track players with the aid of the basketball court monitoring. This task can be regarded as a special case of the multiple object tracking (MOT) problem. But different from it, the task puts request to the existing MOT methods with both good accuracy and operational efficiency (toward real time) in the new basketball scenario. To deal with this task, we make the following attempts: (1) Considering the differences between pedestrians and basketball players and the lack of corresponding dataset for basketball players tracking, we construct a new MOT dataset under the basketball court monitoring scene, to better evaluate MOT methods in our task and to help promote future research in the related task, (2) evaluating the performance of the several candidate MOT methods on the new dataset, and (3) proposing the issues to be further addressed in this specific scenario.

Discriminative and Robust Online Learning for Siamese Visual Tracking

The problem of visual object tracking has traditionally been handled by variant tracking paradigms, either learning a model of the object's appearance exclusively online or matching the object with the target in an offline-trained embedding space. Despite the recent success, each method agonizes over its intrinsic constraint. The online-only approaches suffer from a lack of generalization of the model they learn thus are inferior in target regression, while the offline-only approaches (e.g., convolutional siamese trackers) lack the target-specific context information thus are not discriminative enough to handle distractors, and robust enough to deformation. Therefore, we propose an online module with an attention mechanism for offline siamese networks to extract target-specific features under L2 error. We further propose a filter update strategy adaptive to treacherous background noises for discriminative learning, and a template update strategy to handle large target deformations for robust learning. Effectiveness can be validated in the consistent improvement over three siamese baselines: SiamFC, SiamRPN++, and SiamMask. Beyond that, our model based on SiamRPN++ obtains the best results over six popular tracking benchmarks and can operate beyond real-time.

Parallelization of the Honeybee Search Algorithm for Object Tracking

Object tracking refers to the relocation of specific objects in consecutive frames of a video sequence. Presently, this visual task is still considered an open research issue, and the computer science community attempted solutions from the standpoint of methodologies, algorithms, criteria, benchmarks, and so on. This article introduces a GPU-parallelized swarm algorithm, called the Honeybee Search Algorithm (HSA), which is a hybrid algorithm combining swarm intelligence and evolutionary algorithm principles, and was previously designed for three-dimensional reconstruction. This heuristic inspired by the search for food of honeybees, and here adapted to the problem of object tracking using GPU parallel computing, is extended from the original proposal of HSA towards video processing. In this work, the normalized cross-correlation (NCC) criteria is used as the fitness function. Experiments using 314 video sequences of the ALOV benchmark provides evidence about the quality regarding tracking accuracy and processing time. Also, according to these experiments, the proposed methodology is robust to high levels of Gaussian noise added to the image frames, and this confirms that the accuracy of the original NCC is preserved with the advantage of acceleration, offering the possibility of accelerating latest trackers using this methodology.

Real Time Face Recognition and Tracking System with Multi-Camera Arrangement

Object tracking and face reorganization has received tremendous attention in the video processing community due to its various applications in video surveillance, traffic monitoring and so on. A single camera is not capable to scan 3d view of specified space. So, we use multiple cameras, placed in different sections of the area with overlapping region in field of view (FOV). Every camera will capture the video scene of itself FOV. The system is able to track human successfully by setting up correspondence between objects captured in multiple cameras. Thus, it saves the hectic job of manual tracking. There is a search window available for each object that gives the object’s trajectory. Tracking of object will be given by continuation of this process. For monitoring objects in areas like car parking, banks, hotels etc for security purpose, this system is best. Over the last couple of years, many algorithms and results have been presented for the problem of object tracking and recently the focus has been concentrated on real time person tracking with multiple cameras. Secondly, face detection is one of the best ways of identification. The main applications of automated face recognition are of biometric authentications and surveillances. Face recognition systems has became popular in biometric field as it is non intrusive and does not require the human interference. Up to that, there is no solution or technique that provides robust methods. This paper presents the detection of the face of the person, recognize and do tracking with use of multiple web cameras. Generally in daily camera security systems, cameras have been continuously remain on and large data storage is required in the system. In this real time object tracking system, Infrared sensors are used which indicates presence of person or object. Cameras will turn on only when object is detected by sensor after then the face recognition is carried out. It has capability of high speed processing and achieved low computational requirements. In similar areas, efficiency, accuracy, and speed of identification are the main tackled issues.

Two-stage particle filtering for non-Gaussian state estimation with fading measurements

In this paper, the filtering problem for systems with fading measurements is considered. Taking advantage of the cascaded structure of the system, the original filtering problem is decomposed into two subproblems: 1) recovery of the original measurements from the faded ones, and 2) state estimation based on the recovered signals. A two-stage particle filtering (PF) algorithm is proposed to achieve simultaneous measurement recovery and state estimation. In our scheme, the second-stage resampling procedure can be implemented concurrently, resulting in a significant reduction of execution time. Two examples are provided to demonstrate the effectiveness of our algorithm. A benchmark problem for nonlinear filtering is tackled in the first example and, in the second example, the proposed algorithm is applied to the object tracking problem where the measured signal is distorted by the communication channel. Simulation results show that, compared with the brute force PF, the two-stage PF can strike better balance between tracking accuracy and execution time.

Fast Scale Estimation Method in Object Tracking

Accurate and fast scale estimation of targets is a challenging research problem in visual object tracking. Most trackers employ an exhaustive scale search to estimate the target size. The exhaustive search strategy is computationally expensive and it struggles when encountered with large scale variations. In this paper, a scale adaptive method is proposed, which not only improves the tracking performance, but also greatly reduces the computational costs and improves the tracking speed. Based on the scale estimation method of SAMF, the original 7 fixed scale sizes were reduced to 3, and an adaptive scale size was added. Three fixed scales were used to determine the direction of scale change, and the APCE change rate of the current frame and the previous frame was used to control another adaptive scale size. Finally, the optimal scale estimation was determined. Additionally, we investigate the update strategy to further improve the tracking accuracy. Extensive experiments on OTB50, OTB100 and VOT-ST2019 datasets demonstrate that the proposed method can tackle challenging videos well compared with baseline tracker. On OTB, we obtain a gain of 7.0% in Distance Precision, and 18.8% in Centre Location Error on the selected 43 videos with scale variation attribute, and a mean gain of 6.2% in Precision and 4.6% in Success plots on OTB50, compared with the baseline tracker SAMF. Furthermore, the proposed approach improves the tracking speed by 34% in FPS compared with SAMF.

Visual-LiDAR Based 3D Object Detection and Tracking for Embedded Systems

In recent years, persistent news updates on autonomous vehicles and the claims of companies entering the space, brace the notion that vehicular autonomy of level 5 is just around the corner. However, the main hindrance in asserting the full autonomy still boils down to environmental perception that affects the autonomous decisions. An efficient perceptual system requires redundancy in sensor modalities capable of performing in varying environmental conditions, and providing a reliable information using limited computational resources. This work addresses the task of 3D object detection and tracking in the vehicles’ environment, using camera and 3D LiDAR as primary sensors. The proposed framework is designed to operate in an embedded system that visually classifies the objects using a lightweight neural network, while tracking is performed in 3D space using LiDAR information. The main contribution of this work is 3D LiDAR point cloud classification using visual object detector, and an IMM-UKF-JPDAF based object tracker that jointly performs 3D object detection and tracking. The performance evaluation is carried out using MOT16 metrics and ground truths provided by KITTI Datasets. Furthermore, the proposed tracker is evaluated and compared with state-of-the-art approaches. The experiments suggest that the proposed framework offers a suitable solution for embedded systems to solve 3D object detection and tracking problem, with added benefits.

Adaptive Kernel Correlation Filter Tracking Algorithm in Complex Scenes

The traditional kernel correlation filter (KCF) algorithm has poor tracking results in complex scenes with severe occlusion, deformation, and low resolution and cannot achieve long-term tracking. To improve the accuracy of the tracking algorithm in complex scenes, an adaptive kernel correlation filter algorithm is proposed. First, a multifeature complementary scheme is proposed that linearly weights the responses of the histogram of oriented gradient (HOG) features and color features and learns a target position estimation model to realize target position estimation. Then, an adaptive scale model for estimating the scale transformation of the target is learned by extracting the HOG features of the object. Finally, according to occlusion judgment criteria, the Kalman filter is introduced to correct the position of the tracking target. The accuracy and success rate of the proposed algorithm are verified by simulation analysis on TC-128/OTB2015 benchmarks. Extensive experimental results illustrate that the proposed tracker achieves competitive performance compared with state-of-the-art trackers. The distance precision rate and overlap success rate of the proposed algorithm on OTB2015 are 0.899 and 0.635, respectively. The proposed algorithm effectively solves the long-term object tracking problem in complex scenes. This study provides references for computer vision processing, such as image retrieval, behavior analysis, and intelligent driving.

Deep learning in video multi-object tracking: A survey

The problem of Multiple Object Tracking (MOT) consists in following the trajectory of different objects in a sequence, usually a video. In recent years, with the rise of Deep Learning, the algorithms that provide a solution to this problem have benefited from the representational power of deep models. This paper provides a comprehensive survey on works that employ Deep Learning models to solve the task of MOT on single-camera videos. Four main steps in MOT algorithms are identified, and an in-depth review of how Deep Learning was employed in each one of these stages is presented. A complete experimental comparison of the presented works on the three MOTChallenge datasets is also provided, identifying a number of similarities among the top-performing methods and presenting some possible future research directions.

Object Tracking in Satellite Videos by Improved Correlation Filters With Motion Estimations

As a new method of Earth observation, video satellite is capable of monitoring specific events on the Earth's surface continuously by providing high-temporal resolution remote sensing images. The video observations enable a variety of new satellite applications such as object tracking and road traffic monitoring. In this article, we address the problem of fast object tracking in satellite videos, by developing a novel tracking algorithm based on correlation filters embedded with motion estimations. Based on the kernelized correlation filter (KCF), the proposed algorithm provides the following improvements: 1) proposing a novel motion estimation (ME) algorithm by combining the Kalman filter and motion trajectory averaging and mitigating the boundary effects of KCF by using this ME algorithm and 2) solving the problem of tracking failure when a moving object is partially or completely occluded. The experimental results demonstrate that our algorithm can track the moving object in satellite videos with 95% accuracy.

Fully Convolutional Single-Crop Siamese Networks for Real-Time Visual Object Tracking

The visual object tracking problem seeks to track an arbitrary object in a video, and many deep convolutional neural network-based algorithms have achieved significant performance improvements in recent years. However, most of them do not guarantee real-time operation due to the large computation overhead for deep feature extraction. This paper presents a single-crop visual object tracking algorithm based on a fully convolutional Siamese network (SiamFC). The proposed algorithm significantly reduces the computation burden by extracting multiple scale feature maps from a single image crop. Experimental results show that the proposed algorithm demonstrates superior speed performance in comparison with that of SiamFC.

COB method with online learning for object tracking

Object tracking is a problem about semi-supervised learning with insufficient data set. In the field of military navigation and security of public life, it is widely used to take the place of human beings. In this paper, we come up with a new algorithm based on Bayesian, CNN and PLK optical flow, which is called COB method, for object tracking problems. With the idea of track-by-detect, we cascade CNN after PLK optical flow and integrate them in a Bayesian method. Most importantly our method is proposed with an adaptive integrating method to reduce the influence of over-fitting. The integrator also introduces the competition mechanism between tracker and detector, so that the algorithm is able to update the classifier with online learning. Besides, the regularization of deep learning is used to solve the blind spots of classifier. The experimental results show that the algorithm is more robust than the previous work.

Convolutional Neural Network for Monocular Vision-based Multi-target Tracking

This paper addresses multi-target tracking using a monocular vision sensor. To overcome the fundamental observability issue of the monocular vision, a convolutional neural network (CNN)-based method is proposed. The method combines a CNN-based multi-target detection into a model-based multi-target tracking framework. While previous CNN applications to image-based object recognition and tracking focused on prediction of region of interest (RoI), the proposed method allows for prediction of the three-dimensional position information of the moving objects of interest. This is achieved by appropriately construct a network tailored to the moving object tracking problems with potentially occluded objects. In addition, the cubature Kalman filter integrated with a data association scheme is adopted for effective tracking of nonlinear motion of the objects with the measurements information from the learned network. A virtual simulator that generates the trajectories of the target motions and a sequence of images of the scene has been developed and used to test and verify the proposed CNN scheme. Simulation case studies demonstrate that the proposed CNN improves the position accuracy in the depth direction substantially.