Efficient Visual Tracking Research Articles

An Efficient Visual Tracking System Based on Extreme Learning Machine in the Defence and Military Sector

Visual tracking is the capacity to estimate or forecast a target object’s location in each frame of a video after specifying its starting position. Visual tracking is of essential relevance in defence and military operations. The military can use it to improve situational awareness, improve precise targeting, acquire intelligence in real-time, and efficiently respond to a variety of threats and circumstances. In the past, object tracking systems have relied mostly on algorithms based on deep learning techniques and these tracking algorithms are lacking in both accuracy and speed. In this research, an Extreme Learning Machine-based visual tracking system has been proposed that incorporates properties like high accuracy, low training time, and less network computing complexity as compared to existing deep learning-based tracking algorithms. The Haar wavelet transform is utilized in the recommended technique for feature learning, while the extreme learning machine is utilised for classification and recognition. A benchmark dataset object tracking benchmark-2013 has been used to carry out the experiments. The experiment values indicated that the proposed technique has accomplished enhanced performance over another tracking model. Additionally, we tested the proposed method’s accuracy and robustness regarding certain visual characteristics: Illumination variation, occlusion, deformation, out-of-plane rotation, background clutters, and in-plane rotation. The findings of the simulation revealed that the objects in videos have been 84% accurately tracked by the suggested method.

Target Focused Shallow Transformer Framework for Efficient Visual Tracking

Template learning transformer trackers have achieved significant performance improvement recently due to the longdependency learning using the self-attention (SA) mechanism. However, the typical SA mechanisms in transformers adopt a less discriminative design approach which is inadequate for focusing on the most important target information during tracking. Therefore, existing trackers are easily distracted by background information and have constraints in handling tracking challenges. The focus of our research is to develop a target-focused discriminative shallow transformer tracking framework that can learn to distinguish the target from the background and enable accurate tracking with fast speed. Extensive experiments will be performed on several popular benchmarks, including OTB100, UAV123, GOT10k, LaSOT, and TrackingNet, to demonstrate the effectiveness of the proposed framework.

Efficient Visual Tracking Based on Fuzzy Inference for Intelligent Transportation Systems

Remote monitoring is an important application of intelligent transportation systems (ITSs). The combination of monitoring equipment and tracking algorithms can be used to automatically track moving targets. The tracking algorithm based on the Siamese network is both accurate and efficient, and its development potential is better than that of other algorithms. Its output is a detection map that reflects the probability that any position in the search area is the center of the target’s bounding box, and the maximum value of the detection map is the center of the target’s bounding box predicted by the algorithm. Owing to partial occlusion, target deformation, out-of-view, and background clutter, local maxima in the detection map may also be the center of the target’s bounding box. A tracker’s ability to make accurate judgments is currently limited. Furthermore, previous trackers extracted only the target features in the initial frame as the matching template. Although this matching template is highly reliable, it cannot effectively combine the target features available in the subsequent frames. Therefore, in this study, fuzzy inference is introduced into the tracking process to analyze the reliability of the detection map. When this map is reliable, the target feature of the search area is transformed into a substitute template; otherwise, multiple substitute templates are selected from the template pool for parallel matching as per the set rules. The optimal result is selected from multiple detection results, based on the priority of the detection results when the initial frame is used as the matching template. Experimental results on multiple datasets show that the proposed algorithm is superior to other similar algorithms in terms of multiple assessment metrics and can improve the robustness of remote monitoring tasks in ITSs.

Efficient and Lightweight Visual Tracking with Differentiable Neural Architecture Search

Over the last decade, Siamese network architectures have emerged as dominating tracking paradigms, which have led to significant progress. These architectures are made up of a backbone network and a head network. The backbone network comprises two identical feature extraction sub-branches, one for the target template and one for the search candidate. The head network takes both the template and candidate features as inputs and produces a local similarity score for the target object in each location of the search candidate. Despite promising results that have been attained in visual tracking, challenges persist in developing efficient and lightweight models due to the inherent complexity of the task. Specifically, manually designed tracking models that rely heavily on the knowledge and experience of relevant experts are lacking. In addition, the existing tracking approaches achieve excellent performance at the cost of large numbers of parameters and vast amounts of computations. A novel Siamese tracking approach called TrackNAS based on neural architecture search is proposed to reduce the complexity of the neural architecture applied in visual tracking. First, according to the principle of the Siamese network, backbone and head network search spaces are constructed, constituting the search space for the network architecture. Next, under the given resource constraints, the network architecture that meets the tracking performance requirements is obtained by optimizing a hybrid search strategy that combines distributed and joint approaches. Then, an evolutionary method is used to lighten the network architecture obtained from the search phase to facilitate deployment to devices with resource constraints (FLOPs). Finally, to verify the performance of TrackNAS, comparison and ablation experiments are conducted using several large-scale visual tracking benchmark datasets, such as OTB100, VOT2018, UAV123, LaSOT, and GOT-10k. The results indicate that the proposed TrackNAS achieves competitive performance in terms of accuracy and robustness, and the number of network parameters and computation volume are far smaller than those of other advanced Siamese trackers, meeting the requirements for lightweight deployment to resource-constrained devices.

Dynamic Target Tracking and Ingressing of a Small UAV Using Monocular Sensor Based on the Geometric Constraints

In many applications of airborne visual techniques for unmanned aerial vehicles (UAVs), lightweight sensors and efficient visual positioning and tracking algorithms are essential in a GNSS-denied environment. Meanwhile, many tasks require the ability of recognition, localization, avoiding, or flying pass through these dynamic obstacles. In this paper, for a small UAV equipped with a lightweight monocular sensor, a single-frame parallel-features positioning method (SPPM) is proposed and verified for a real-time dynamic target tracking and ingressing problem. The solution is featured with systematic modeling of the geometric characteristics of moving targets, and the introduction of numeric iteration algorithms to estimate the geometric center of moving targets. The geometric constraint relationships of the target feature points are modeled as non-linear equations for scale estimation. Experiments show that the root mean square error percentage of static target tracking is less than 1.03% and the root mean square error of dynamic target tracking is less than 7.92 cm. Comprehensive indoor flight experiments are conducted to show the real-time convergence of the algorithm, the effectiveness of the solution in locating and tracking a moving target, and the excellent robustness to measurement noises.

Incremental visual tracking via sparse discriminative classifier

Currently, visual object tracking is a core research area as it can be applied in many applications of computer vision. However, tracking of a visual object is a difficult task as it can go through different varying conditions like occlusion of the target object, appearance variation, illumination variation, etc. during the tracking process. An efficient and robust visual object tracking based on sparse discriminative classier (SDC) and principal component analysis (PCA) subspace representation is presented in this work. The PCA subspace representation modelled the appearance model of the target object and SDC separates the target object and background object very efficiently. The computational complexity is much better than the other existing methods in the literature. Both quantitative and qualitative analyses of different video sequences are done to compare the proposed tracking algorithm with the other existing tracking algorithms. The experimental results show that the proposed method outperforms the other existing tracking algorithms.

An efficient hybrid framework for visual tracking using Exponential Quantum Particle Filter and Mean Shift optimization

Visual object tracking is a key component in many computer vision applications. In real time visual tracking, abrupt changes in speed and direction of the object are demanding challenges. In this paper, we present an efficient visual tracking framework which can efficiently handle the above challenge, i.e., abrupt motion of the object. The framework is formulated by hybridizing the proposed Exponential Quantum Particle Filter (EQPF) with the traditional Mean-Shift (MS) optimization for efficient computation in object tracking task. The efficacy of EQPF in estimating the abrupt changes in functions, is tested on the standard Quail function, and then it has been successfully applied in object tracking algorithm. The effective multi-modal propagation strategies of Quantum Particle Filter (QPF) enables the tracker to handle the abrupt changes in speed and direction, whereas, the hybridization with MS enhances the computational efficiency by reducing the number of particles. Performance of the proposed method is assessed by experimenting on different publicly available challenging sequences. Both the subjective and objective evaluations are carried out to validate the superiority of the proposed tracking method over other state-of-the-art methods.

POST: POlicy-Based Switch Tracking

In visual object tracking, by reasonably fusing multiple experts, ensemble framework typically achieves superior performance compared to the individual experts. However, the necessity of parallelly running all the experts in most existing ensemble frameworks heavily limits their efficiency. In this paper, we propose POST, a POlicy-based Switch Tracker for robust and efficient visual tracking. The proposed POST tracker consists of multiple weak but complementary experts (trackers) and adaptively assigns one suitable expert for tracking in each frame. By formulating this expert switch in consecutive frames as a decision-making problem, we learn an agent via reinforcement learning to directly decide which expert to handle the current frame without running others. In this way, the proposed POST tracker maintains the performance merit of multiple diverse models while favorably ensuring the tracking efficiency. Extensive ablation studies and experimental comparisons against state-of-the-art trackers on 5 prevalent benchmarks verify the effectiveness of the proposed method.

Efficient Visual Tracking With Stacked Channel-Spatial Attention Learning

Template based learning, particularly Siamese networks, has recently become popular due to balancing accuracy and speed. However, preserving tracker robustness against challenging scenarios with real-time speed is a primary concern for visual object tracking. Siamese trackers confront difficulties handling target appearance changes continually due to less discrimination ability learning between target and background information. This paper presents stacked channel-spatial attention within Siamese networks to improve tracker robustness without sacrificing fast-tracking speed. The proposed channel attention strengthens target-specific channels increasing their weight while reducing the importance of irrelevant channels with lower weights. Spatial attention is focusing on the most informative region of the target feature map. We integrate the proposed channel and spatial attention modules to enhance tracking performance with end-to-end learning. The proposed tracking framework learns what and where to highlight important target information for efficient tracking. Experimental results on widely used OTB100, OTB50, VOT2016, VOT2017/18, TC-128, and UAV123 benchmarks verified the proposed tracker achieved outstanding performance compared with state-of-the-art trackers.

Adversarial Feature Sampling Learning for Efficient Visual Tracking

The tracking-by-detection tracking framework usually consists of two stages: drawing samples around the target object and classifying each sample as either the target object or background. Current popular trackers under this framework typically draw many samples from the raw image and feed them into the deep neural networks, resulting in high computational burden and low tracking speed. In this article, we propose an adversarial feature sampling learning (AFSL) method to address this problem. A convolutional neural network is designed, which takes only one cropped image around the target object as input, and samples are collected from the feature maps with spatial bilinear resampling. To enrich the appearance variations of positive samples in the feature space, which has limited spatial resolution, we fuse the high-level features and low-level features to better describe the target by using a generative adversarial network. Extensive experiments on benchmark data sets demonstrate that the proposed ASFL achieves leading tracking accuracy while significantly accelerating the speed of tracking-by-detection trackers.

One-Shot Scale and Angle Estimation for Fast Visual Object Tracking

Visual object tracking is the process of tracking an arbitrary object in a video, where the bounding box of the object is given in the first frame. Siamese network-based visual object tracking approaches have recently received considerable attention for their high speed and superior performance. However, for scale and angle estimation, Siamese trackers require multiple search regions, which increases the computation time, thereby decreasing the real-time tracking performance. This paper proposes a one-shot Siamese network, named Siam-OS, for fast and efficient visual object tracking. Siam-OS uses only a single search region and estimates the scale and angle of the target bounding box. This significantly reduces the number of computations required for the deep convolutional feature extraction, and thus increases the tracking speed. The experimental results with Visual Object Tracking (VOT) benchmarks show the effectiveness of the proposed Siam-OS in terms of the accuracy, robustness, expected average overlap, and speed.

Efficient visual tracking using multi-feature regularized robust sparse coding and quantum particle filter based localization

Visual object tracking is a challenging task in the field of computer vision, due to many constraints like scene variation, occlusion, cluttered background and higher data size. The sparse representation tool of the compressive sensing theory, turns out to be an effective way to implement the object tracking algorithm with less computational load and at a faster speed. In this paper, we have proposed an effective object tracking framework by using a regularized robust sparse coding (RRSC) for representing the multi-feature templates of the candidate objects. Moreover, an efficient Quantum Particle Filter (QPF) based Bayesian state estimation for tracking is also proposed for localizing the target object in the subsequent frames. The RRSC assures robustness to occlusion and noise, while the QPF successfully deals with the abrupt motion of the object. The Local Binary Pattern (LBP) feature and the Ohta color feature, reconciled in appearance modelling enhance the discriminant description. Both subjective as well as objective evaluation of the proposed tracking method is carried out for validating its efficacy in comparison to the other state-of-the-art methods. The evaluation is executed on different publicly available data sets, which illustrates the superiority of the proposed method.

A triangular radial cubic spline deformation model for efficient 3D beating heart tracking

A novel deformable model is proposed for efficient 3D visual tracking of beating heart. The model is parameterized by the 3D coordinates of four control points: the three vertices and the circumcenter of a triangular target region. Nonlinear deformation on heart surfaces is handled by cubic spline interpolation based on radial pixel distances from the circumcenter. With a pre-computable design matrix, the model can be represented efficiently by a simple matrix equation. An iterative algorithm is developed based on the efficient second-order minimization to compute model parameters at each frame. The proposed tracking method is validated on the stereo-endoscopic videos of phantom heart and in vivo heart that are recorded by the da Vinci\(^{\tiny \textregistered }\) surgical system.

Convolutional neural net bagging for online visual tracking

Recently, Convolutional Neural Nets (CNNs) have been successfully applied to online visual tracking. However, a major problem is that such models may be inevitably over-fitted due to two main factors. The first one is the label noise because the online training of any models relies solely on the detection of the previous frames. The second one is the model uncertainty due to the randomized training strategy. In this work, we cope with noisy labels and the model uncertainty within the framework of bagging (bootstrap aggregating), resulting in efficient and effective visual tracking. Instead of using multiple models in a bag, we design a single multitask CNN for learning effective feature representations of the target object. In our model, each task has the same structure and shares the same set of convolutional features, but is trained using different random samples generated for different tasks. A significant advantage is that the bagging overhead for our model is minimal, and no extra efforts are needed to handle the outputs of different tasks as done in those multi-lifespan models. Experiments demonstrate that our CNN tracker outperforms the state-of-the-art methods on three recent benchmarks (over 80 video sequences), which illustrates the superiority of the feature representations learned by our purely online bagging framework.

Integrating Local Binary Patterns into Normalized Moment of Inertia for Updating Tracking Templates

This paper presents an efficient visual tracking framework which is robust to rotation, scale variation and occlusion. The target template is characterized by Local binary patterns (LBP), which exhibit invariance to rotation. The LBP features are then integrated into the Normalized moment of inertia (NMI) to decide whether the template requires update. This procedure enables an adaptive template matching strategy which addresses the tracking failures arising from scale variations. Kalman filtering is exploited for predicting the trajectory of the target when it is occluded. The matching efficiency is achieved by applying a locally pyramid searching scheme. Experimental results validate the efficiency and effectiveness of our tracking framework.

Effective real-time vehicle tracking using discriminative sparse coding on local patches

A visual tracking framework that provides an object detector and tracker, which focuses on effective and efficient visual tracking in surveillance of real-world intelligent transport system applications, is proposed. The framework casts the tracking task as problems of object detection, feature representation, and classification, which is different from appearance model-matching approaches. Through a feature representation of discriminative sparse coding on local patches called DSCLP, which trains a dictionary on local clustered patches sampled from both positive and negative datasets, the discriminative power and robustness has been improved remarkably, which makes our method more robust to a complex realistic setting with all kinds of degraded image quality. Moreover, by catching objects through one-time background subtraction, along with offline dictionary training, computation time is dramatically reduced, which enables our framework to achieve real-time tracking performance even in a high-definition sequence with heavy traffic. Experiment results show that our work outperforms some state-of-the-art methods in terms of speed, accuracy, and robustness and exhibits increased robustness in a complex real-world scenario with degraded image quality caused by vehicle occlusion, image blur of rain or fog, and change in viewpoint or scale.

Multi-Target Tracking and Behavior Analysis Method for Video Surveillance Applications

In order to obtain a satisfactory performance of visual tracking and video surveillance in complex dynamic scenes without the supervision of qualified workers, an efficient visual detection and tracking method is proposed, which can realize target counting and surveillance, behavior analysis and abnormal detection. Multi-targets tracking method based on novel Bayesian tracking model can manage multimodal distributions without explicitly computing the association between tracked targets and detections. The proposed algorithm is compared with recent works, which shows that it is robust to erroneous, distorted and missing detections and it can be applied in security and management of access points.

Joint Localization of Pursuit Quadcopters and Target Using Monocular Cues

Pursuit robots (autonomous robots tasked with tracking and pursuing a moving target) require accurate tracking of the target’s position over time. One possibly effective pursuit platform is a quadcopter equipped with basic sensors and a monocular camera. However, the combined noise in the quadcopter’s sensors causes large disturbances in the target’s 3D position estimate. To solve this problem, in this paper, we propose a novel method for joint localization of a quadcopter pursuer with a monocular camera and an arbitrary target. Our method localizes both the pursuer and target with respect to a common reference frame. The joint localization method fuses the quadcopter’s kinematics and the target’s dynamics in a joint state space model. We show that predicting and correcting pursuer and target trajectories simultaneously produces better results than standard approaches to estimating relative target trajectories in a 3D coordinate system. Our method also comprises a computationally efficient visual tracking method capable of redetecting a temporarily lost target. The efficiency of the proposed method is demonstrated by a series of experiments with a real quadcopter pursuing a human. The results show that the visual tracker can deal effectively with target occlusions and that joint localization outperforms standard localization methods.

Robust and efficient visual tracking under illumination changes based on maximum color difference histogram and min-max-ratio metric

Visual tracking under illumination changes is a challenging task for numerous computer vision applications. We propose a robust and efficient tracking algorithm based on maximum color difference histogram (MCDH) and a well-designed min-max-ratio (MMR) similarity metric. Appearance model is essential to the tracker’s robustness under illumination changes. We propose a new feature descriptor MCDH, calculated by exploiting the maximum color difference information within the eight-neighborhood of each pixel in the object region, to build the object appearance model. Unlike the traditional histogram-based algorithms, the MCDH is efficiently extracted by employing the local integral histogram which is propagated in a specially designed local image region. The similarity metric plays an important role on accurately locating the target. However, the existing metrics are not suitable for comparisons of MCDHs due to many zero-valued bins in MCDH. Therefore, we propose a new MMR metric, defined as the average ratio between the minimum and maximum of a MCDH bin pair. The combination of proposed components enables the tracker to be robust to illumination changes with high computational efficiency. Experiments demonstrate superior performance of the proposed tracking algorithm compared with 10 state-of-art tracking methods when illumination varies.

Particle Probability-hypothesis-density Filter with Kernel Based State Extraction for Efficient Multi-target Visual Tracking

Particle Probability-hypothesis-density Filter with Kernel Based State Extraction for Efficient Multi-target Visual Tracking