Visual Tracking Method Research Articles

An accelerated correlation filter tracker

Recent visual object tracking methods have witnessed a continuous improvement in the state-of-the-art with the development of efficient discriminative correlation filters (DCF) and robust deep neural network features. Despite the outstanding performance achieved by the above combination, existing advanced trackers suffer from the burden of high computational complexity of the deep feature extraction and online model learning. We propose an accelerated ADMM optimisation method obtained by adding a momentum to the optimisation sequence iterates, and by relaxing the impact of the error between DCF parameters and their norm. The proposed optimisation method is applied to an innovative formulation of the DCF design, which seeks the most discriminative spatially regularised feature channels. A further speed up is achieved by an adaptive initialisation of the filter optimisation process. The significantly increased convergence of the DCF filter is demonstrated by establishing the optimisation process equivalence with a continuous dynamical system for which the convergence properties can readily be derived. The experimental results obtained on several well-known benchmarking datasets demonstrate the efficiency and robustness of the proposed ACFT method, with a tracking accuracy comparable to the start-of-the-art trackers.

Object-adaptive LSTM network for real-time visual tracking with adversarial data augmentation

In recent years, deep learning based visual tracking methods have obtained great success owing to the powerful feature representation ability of Convolutional Neural Networks (CNNs). Among these methods, classification-based tracking methods exhibit excellent performance while their speeds are heavily limited by the expensive computation for massive proposal feature extraction. In contrast, matching-based tracking methods (such as Siamese networks) possess remarkable speed superiority. However, the absence of online updating renders these methods unadaptable to significant object appearance variations. In this paper, we propose a novel real-time visual tracking method, which adopts an object-adaptive LSTM network to effectively capture the video sequential dependencies and adaptively learn the object appearance variations. For high computational efficiency, we also present a fast proposal selection strategy, which utilizes the matching-based tracking method to pre-estimate dense proposals and selects high-quality ones to feed to the LSTM network for classification. This strategy efficiently filters out some irrelevant proposals and avoids the redundant computation for feature extraction, which enables our method to operate faster than conventional classification-based tracking methods. In addition, to handle the problems of sample inadequacy and class imbalance during online tracking, we adopt a data augmentation technique based on the Generative Adversarial Network (GAN) to facilitate the training of the LSTM network. Extensive experiments on four visual tracking benchmarks demonstrate the state-of-the-art performance of our method in terms of both tracking accuracy and speed, which exhibits great potentials of recurrent structures for visual tracking.

Visual contour tracking based on inner-contour model particle filter under complex background

In this paper, a novel particle filter–based visual contour tracking method is proposed, which uses inner-contour model to track contour object under complex background. The purpose is to achieve effectiveness and robustness against complex background. To that end, the proposed method first utilized Sobel edge detector to detect the edge information along the normal line of the contour. Then, it sampled the inner part of the normal line to get the local color information, which was then combined with the edge information to construct new normal line likelihood. After that, all the inner color information was used to construct global color likelihood. Finally, the edge information, local color information, and global color information were fused into new observation likelihood. Experimental results showed that the proposed method was robust for contours tracking under complex background, and it was also computationally efficient and can run in real-time completely.

Anti-occlusion object tracking based on correlation filter

Despite remarkable progress, visual object tracking is still a challenging task as objects usually suffer from significant appearance changes, fast motion, and serious occlusion. In this paper, we propose an anti-occlusion correlation filter-based tracking method (AO-CF) for robust visual tracking. We first propose an occlusion criterion based on continuous response values. Based on the criterion, objects are divided into four categories to adaptively identify the occlusion of objects. Then we propose a new detection condition for detecting proposals. When the occlusion criterion is triggered, the re-detection mechanism is executed and the tracker is commanded to stop, and then the re-detector selects the most reliable proposal to reinitialize the tracker. Experimental results show that our method outperforms other state-of-the-art trackers in terms of both precision rate and success rate on the widely used object tracking benchmark dataset. In addition, AO-CF is able to achieve real-time tracking speed.

Incremental learning-based visual tracking with weighted discriminative dictionaries

Existing sparse representation-based visual tracking methods detect the target positions by minimizing the reconstruction error. However, due to complex background, illumination change, and occlusion problems, these methods are difficult to locate the target properly. In this article, we propose a novel visual tracking method based on weighted discriminative dictionaries and a pyramidal feature selection strategy. First, we utilize color features and texture features of the training samples to obtain multiple discriminative dictionaries. Then, we use the position information of those samples to assign weights to the base vectors in dictionaries. For robust visual tracking, we propose a pyramidal sparse feature selection strategy where the weights of base vectors and reconstruction errors in different feature are integrated together to get the best target regions. At the same time, we measure feature reliability to dynamically adjust the weights of different features. In addition, we introduce a scenario-aware mechanism and an incremental dictionary update method based on noise energy analysis. Comparison experiments show that the proposed algorithm outperforms several state-of-the-art methods, and useful quantitative and qualitative analyses are also carried out.

Application of Automatic Kinematic Analysis Program for the Evaluation of Dysphagia in ALS patients

Dysphagia in amyotrophic lateral sclerosis (ALS) increases the risk of malnutrition, dehydration, and aspiration pneumonia. Kinematic analysis of videofluoroscopic swallowing study (VFSS) can provide detailed movement of the hyoid bone, revealing abnormalities of swallowing in ALS patients. We developed an automated kinematic analysis program (AKAP) that analyzes the trajectory of the hyoid bone via a visual tracking method. The aim of this study was to investigate the hyoid movement in ALS patients using AKAP and compare it with non-dysphagic subjects. Thirty ALS patients who underwent VFSS in Seoul National University Bundang Hospital between 2015 and 2017 were recruited. For comparison, 30 age-matched control subjects were also enrolled; the same swallowing study was conducted using thin fluid and yogurt. The hyoid bone movement was analyzed by evaluating the vertical and horizontal distances with four peak points (A, B, C, D), and the time of each point were also calculated. With respect to distance parameters, only vertical peak distance (distance between B, D points) during thin fluid swallowing was significantly decreased in ALS patients. (p = 0.038) With respect to temporal parameters, Time ABC, Time ABCD, and Duration C were significantly increased in ALS patients when swallowing both thin fluid and yogurt. (Time ABC p = 0.019, p = 0.002; Time ABCD p = 0.001, p = 0.004; Duration C p = 0.004, p = 0.025 respectively). This result revealed that dysphagia in ALS patient is caused by decreased velocity of hyoid bone movement due to the development of weakness in swallowing-related muscles. The parameters of kinematic analysis could be used to quantitatively evaluate dysphagia in motor neuron disease.

Coarse-to-Fine UAV Target Tracking With Deep Reinforcement Learning

The aspect ratio of a target changes frequently during an unmanned aerial vehicle (UAV) tracking task, which makes the aerial tracking very challenging. Traditional trackers struggle from such a problem as they mainly focus on the scale variation issue by maintaining a certain aspect ratio. In this paper, we propose a coarse-to-fine deep scheme to address the aspect ratio variation in UAV tracking. The coarse-tracker first produces an initial estimate for the target object, then a sequence of actions are learned to fine-tune the four boundaries of the bounding box. The coarse-tracker and the fine-tracker are designed to have different action spaces and operating target. The former dominates the entire bounding box and the latter focuses on the refinement of each boundary. They are trained jointly by sharing the perception network with an end-to-end reinforcement learning architecture. Experimental results on benchmark aerial data set prove that the proposed approach outperforms existing trackers and produces significant accuracy gains in dealing with the aspect ratio variation in UAV tracking. Note to Practitioners —During the past years, unmanned aerial vehicle (UAV) have gained much attention for both industrial and consumer uses. It is in urgent demand to endow the UAV with intelligent vision-based techniques, and the automatic target following via visual tracking methods as one of the most fundamental intelligent features could promote various applications of UAVs, such as surveillance, augmented reality, and behavior modeling. Nonetheless, the primary issue of a UAV-based tracking method is the platform itself: it is not stable, it tends to have sudden movements, it generates nonhomogeneous data (scale, angle, rotation, depth, and so on), all of them tend to change the aspect ratio of the target frequently and further increase the difficulty of object tracking. This paper aims to address the aspect ratio change (ARC) problem in UAV tracking. We present a coarse-to-fine strategy for UAV tracking. Specifically, the coarse bounding box is obtained to locate the target firstly. Then, a refinement scheme is performed on each boundary to further improve the position estimate. The tracker is proved to be effective to increase the resistance to the ARC. Such a method can be implemented on UAV to improve the target-following performance.

RFR-DLVT: a hybrid method for real-time face recognition using deep learning and visual tracking

ABSTRACT In this paper, we proposed a hybrid method based on Deep Learning (DL) and visual tracking, RFR-DLVT, to achieve effective face recognition (FR). First, video sequences are divided into reference frames (RFs) and non-reference frames (NRFs). Then, the target face is identified through the DL-based FR method in RFs. In the meantime, the Kernelized-correlation-filters-based visual tracking method is used in NRFs to speed up FR. Our proposed method is tested on common data sets and achieves better performance. Particularly, RFR-DLVT has an accuracy of 99.6% and an efficiency of 30 (FPS) in the real-time FR on real-life surveillance videos.

Dynamic Analysis of Construction Safety Risk and Visual Tracking of Key Factors based on Behavior-based Safety and Building Information Modeling

Construction has long been seen as a high-risk industry. Many studies conduct risk management by controlling construction risk indicators, but few studies associate risk indicators with time and space to propose long-term risk management methods. Therefore, this study proposes a dynamic analysis and visual tracking method based on behavior-based safety (BBS). This study establishes a BBS observation checklist and records workers’ unsafe behavior. The risk level of unsafe behavior is then determined by grey clustering model. When a high risk occurs, an improved grey correlation model is used to track key indicators that affect risk. In order to achieve visual risk management, this study develops semantic logic to predefine the relationship between components and space. In schedule simulation of BIM, the key indicators of BBS are transformed into an executable visual inspection between work items, components, and space through ARC. This method makes it easier for construction managers to combine time and space to manage safety risk and to adopt appropriate strategies in a timely manner to improve the efficiency of safety management.

Visual object tracking based on discriminant DCT features

In this paper, a visual object tracking method is proposed based on sparse 2-dimensional discrete cosine transform (2D DCT) coefficients as discriminative features. To select the discriminative DCT coefficients, we give two propositions. The propositions select the features based on estimated mean of feature distributions in each frame. Some intermediate tracking instances are obtained by (a) computing feature similarity using kernel, (b) finding the maximum classifier score computed using ratio classifier, and (c) combinations of both. Another intermediate tracking instance is obtained using incremental subspace learning method. The final tracked instance amongst the intermediate instances are selected by using a discriminative linear classifier learned in each frame. The linear classifier is updated in each frame using some of the intermediate tracked instances. The proposed method has a better tracking performance as compared to state-of-the-art video trackers in a dataset of 50 challenging video sequences.

Mutual kernelized correlation filters with elastic net constraint for visual tracking

In this paper, we propose a robust visual tracking method based on mutual kernelized correlation filters with elastic net constraint. First, two correlation filters are trained in a general framework jointly in a closed form, which are interrelated and interacted on each other. Second, elastic net constraint is imposed on each discriminative filter, which is able to filter some interfering features. Third, scale estimation and target re-detection scheme are adopted in our framework, which can deal with scale variation and tracking failure effectively. Extensive experiments on some challenging tracking benchmarks demonstrate that our proposed method is able to obtain a competitive tracking performance against other state-of-the-art algorithms.

Robust Estimation of Similarity Transformation for Visual Object Tracking

Most of existing correlation filter-based tracking approaches only estimate simple axis-aligned bounding boxes, and very few of them is capable of recovering the underlying similarity transformation. To tackle this challenging problem, in this paper, we propose a new correlation filter-based tracker with a novel robust estimation of similarity transformation on the large displacements. In order to efficiently search in such a large 4-DoF space in real-time, we formulate the problem into two 2-DoF sub-problems and apply an efficient Block Coordinates Descent solver to optimize the estimation result. Specifically, we employ an efficient phase correlation scheme to deal with both scale and rotation changes simultaneously in log-polar coordinates. Moreover, a variant of correlation filter is used to predict the translational motion individually. Our experimental results demonstrate that the proposed tracker achieves very promising prediction performance compared with the state-of-the-art visual object tracking methods while still retaining the advantages of high efficiency and simplicity in conventional correlation filter-based tracking methods.

Adversarial Deep Tracking

A number of visual tracking methods achieve the state-of-the-art performance based on deep learning recently. However, most of these trackers utilize the deep neural network in regression task or classification task separately. In this paper, we propose an adversarial deep tracking framework. The framework is composed of a fully convolutional Siamese neural network (regression network) and a discriminative classification network. Then, we jointly optimize the regression network and the classification network by adversarial learning. In the uniform framework, the regression network and classification network can be trained end-to-end as a whole using large amounts of video training data sets. During the testing phase, the regression network generates a response map which reflects the location and the size of the target within each candidate search patch, and the classification network discriminates which response map is the best in terms of the corresponding template patch and candidate search patch. In addition, we propose an attention visualization algorithm for our tracker, and it reflects the area that attracts the attention of our tracker during tracking. The experimental results on three large-scale visual tracking benchmarks (OTB-100, TC-128, and VOT2016) demonstrate the effectiveness of the proposed tracking algorithm and show that our tracker performs comparably against the state-of-the-art trackers.

Salient Superpixel Visual Tracking with Graph Model and Iterative Segmentation

Visual object tracking is to locate an object of interest in a sequence of consecutive video frames, which is widely applied in many high-level computer vision tasks such as intelligent video surveillance and robotics. It is of great challenges for visual tracking methods to handle large target appearance variations caused by pose deformation, fast motion, occlusion, and surrounding environments in real-time videos. In this paper, inspired by human attention cognitive saliency model, we propose a visual tracking method based on salient superpixels which integrates the target appearance similarity and cognitive saliency, and helps to location inference and appearance model updating. The saliency of superpixel is detected by graph model and manifold ranking. We cluster the superpixels of the first four target boxes into a set corresponding to object foreground and model the target appearance with color descriptors. While tracking, the relevance is computed between the candidate superpixels and the target appearance set. We also propose an iterative threshold segmentation method to distinguish the foreground and background of superpixels based on saliency and relevance. To increase the accuracy of location inference, we explore particle filter in both confidence estimation and sampling procedures. We compared our method with the existing techniques in OTB100 dataset in terms of precision based on center location error and success rate based on overlap, and the experimental results show that our proposed method achieved substantially better performance. Promising results have shown that the proposed salient superpixel-based approach is effective to deformation, occlusion, and other challenges in object tracking.

Robust visual tracking based on response stability

In this paper, a new approach of response stability based for visual object tracking is developed. This approach proposes a response stability criterion to measure the tracking quality and fuse tracking results of multiple layers of a convolutional neural network (CNN). Inspired by recent detection based methods for visual tracking, the detection capability of EdgeBoxes is investigated, and proposes to re-detect target when tracking failure occurs. In addition, 3D locally adaptive regression kernels (LARK) feature is employed in correlation filter based tracking framework. Extensive experimental results and performance compared with state-of-the-art tracking algorithms on challenging benchmark datasets show that our method is more accurate and robust.

Semantics-Aware Visual Object Tracking

In this paper, we propose a semantics-aware visual object tracking method, which introduces semantics into the tracking procedure and extends the model of an object with explicit semantics prior to enhancing the robustness of three key aspects of the tracking framework, i.e., appearance model, search scheme, and scale adaptation. We first present a semantic object proposal generation method for video sequences to generate high-quality category-oriented object proposals. Then, a hybrid semantics-aware tracking algorithm with semantic compatibility is proposed. This algorithm takes full advantages of globally sparse semantic object proposal prediction and locally dense prediction with a template model and semantic distractor-aware color appearance model. Furthermore, we propose to exploit semantics to localize object accurately via an energy minimization framework-based scale adaptation method, which jointly integrates dense location prior, instance-specific color, and category-specific semantic information. Extensive experiments are conducted on two widely used benchmarks, and the results demonstrate that our method achieves the state-of-the-art performance.

Real-Time Visual Tracking of the HDR Source during Skin Therapy Enabled by Scintillation Markers

The most frequent cause of high dose rate (HDR) brachytherapy medical events is incorrect measurement and/or entering of catheter length in the treatment planning system. Another common cause of medical events is the incorrect connection of catheters, transfer tubes and afterloader channels at the treatment. These errors displace the HDR source from its planned position up to a distance of 30 cm away from the treatment target [1], Resulting in dosimetric errors that under-dose, or completely miss, the treatment target and increase dose to healthy tissue. For the treatment of complex superficial skin lesions, the probability of these errors is increased due to the complex shape of the surface applicator and numerous catheters required for these treatments. Current quality assurance methods aim to prevent these errors from occurring prior to treatment by relying on manual checks and verbal verification between physicians, therapists and physicists. Despite these efforts, there remain a lack of methods to verify the HDR source position during treatment to ensure that radiation is being delivered correctly and prevent misadministration. Here, we developed a method for visual real-time tracking of the source to overcome the inadequacies of the present QA processes.

Robust long‐term correlation tracking with multiple models

To address the challenge of repetitive target appearance variation and frequent occlusion, existing visual tracking methods either handle corrupted samples or correct the appearance model. In this study, the authors propose a novel framework that successfully combines these two strategies. In their method, the base tracker is an improved discriminative correlation filter-based tracker, in which an independent classifier is employed to alleviate the problem of corrupted samples; the best model is selected for improvement from a group of models, which they call a ‘model colony’. The model colony is composed of models updated via different processes. The correlation output and the peak-to-sidelobe ratio are used to evaluate each model in the model colony. In addition, they propose a novel criterion called the maximum-to-others ratio for superior model selection. Experiments on 80 challenging sequences show that their tracker outperforms state-of-the-art trackers. In addition, experimental results demonstrate that their formulation significantly improves the performance of their base tracker.

A Robust Visual Tracking Method for Unmanned Mobile Systems

This paper introduces a robust visual tracking of objects in complex environments with blocking obstacles and light reflection noises. This visual tracking method utilizes a transfer matrix to project image pixels back to real-world coordinates. During the image process, a color and shape test is used to recognize the object and a vector is used to represent the object, which contains the information of orientation and body length of the object. If the object is partially blocked by the obstacles or the reflection from the water surface, the vector predicts the position of the object. During the real-time tracking, a Kalman filter is used to optimize the result. To validate the method, the visual tracking algorithm was tested by tracking a submarine and a fish on the water surface of a water tank, above which three pieces of blur glass were blocking obstacles between the camera and the object. By using this method, the interference from the reflection of the side glass and the fluctuation of the water surface can be also avoided.

Memory Mechanisms for Discriminative Visual Tracking Algorithms With Deep Neural Networks

Deep-neural-networks-based online visual tracking methods have achieved state-of-the-art results. One of the core components of these methods is the memory pool, in which a number of samples consisting of image patches and the corresponding labels are stored to update the online tracking network. Hence, the mechanism of updating the stored samples determines the performance of the tracking method. In this paper, a novel memory mechanism is proposed to control the writing and reading accesses of the memory pool using credit assignment network ${H}$ , which learns features of the target object. This memory mechanism comprises the writing and reading mechanisms. In the writing mechanism, network ${H}$ produces credits for the current tracked object and the samples in the memory pool. This ensures that the reliable samples are written into the memory pool and the unreliable samples are replaced if the memory pool is full. In the reading mechanism, network ${H}$ assigns an importance score to each sample selected to update the online tracking network. The state-of-the-art tracking methods with and without the proposed memory mechanism are evaluated on the CVPR2013 and OTB100 benchmarks. The experimental results demonstrated that the proposed memory mechanism improves tracking performance significantly.