Discriminative Correlation Filter Research Articles

Learning a Context-Aware Environmental Residual Correlation Filter via Deep Convolution Features for Visual Object Tracking

Visual tracking has become widespread in swarm robots for intelligent video surveillance, navigation, and autonomous vehicles due to the development of machine learning algorithms. Discriminative correlation filter (DCF)-based trackers have gained increasing attention owing to their efficiency. This study proposes “context-aware environmental residual correlation filter tracking via deep convolution features (CAERDCF)” to enhance the performance of the tracker under ambiguous environmental changes. The objective is to address the challenges posed by intensive environment variations that confound DCF-based trackers, resulting in undesirable tracking drift. We present a selective spatial regularizer in the DCF to suppress boundary effects and use the target’s context information to improve tracking performance. Specifically, a regularization term comprehends the environmental residual among video sequences, enhancing the filter’s discrimination and robustness in unpredictable tracking conditions. Additionally, we propose an efficient method for acquiring environmental data using the current observation without additional computation. A multi-feature integration method is also introduced to enhance the target’s presence by combining multiple metrics. We demonstrate the efficiency and feasibility of our proposed CAERDCF approach by comparing it with existing methods using the OTB2015, TempleColor128, UAV123, LASOT, and GOT10K benchmark datasets. Specifically, our method increased the precision score by 12.9% in OTB2015 and 16.1% in TempleColor128 compared to BACF.

Anti-Occlusion Visual Tracking Algorithm for UAVs with Multi-Feature Adaptive Fusion

Most of the existing trackers based on discriminative correlation filters use only one feature or a simple linear fusion of multiple features for object tracking, and most of them lack a mechanism to handle occlusions. This leads to poor tracking performance in rapidly changing and easily occluded scenarios, especially on unmanned aerial vehicle (UAV) platforms. To address this issue, this paper proposes an anti-occlusion visual tracking algorithm for UAVs with multi-feature adaptive fusion named multi-feature adaptive fusion and anti-occlusion tracker (MAFAOT). It introduces a novel approach for implementing an adaptive fusion of multiple features. This method transforms the multi-feature fusion problem into a maximization issue by designing a tracking quality evaluation index. It successfully achieves an adaptive fusion of gradient direction histogram and color histogram feature responses. MAFAOT also introduces an anti-occlusion update pool strategy, enabling the tracker to adapt dynamically to various complex scenarios, including occlusion and motion blur. The experimental results on the OTB100 and UAV123 datasets confirm the significant advantages of MAFAOT in terms of precision and success rate compared to other correlation filter-based algorithms. The proposed methods further enhance the expressiveness of the features and effectively avoid the problem of tracker contamination caused by occlusion. Furthermore, this paper applies the proposed methods to the kernelized correlation filters (KCF) algorithm. On the OTB100 dataset, the improved KCF algorithm shows an improvement of 10.94% in precision and 11.11% in success rate. On the UAV123 dataset, it shows an improvement of 14.53% in precision and 16.62% in success rate, further verifying the effectiveness and versatility of the proposed methods.

Visual tracking with pyramidal feature fusion and transformer based model predictor

Discriminative correlation filters (DCF) have achieved much success in visual tracking. However, most of them simply rely on the features extracted by the last layer of the backbone, while ignoring the low-level rich structural information. In addition, they normally minimize the tailored objective functions to predict the target model in a direct way, which introduces inductive bias and limits the expressivity of the trackers. In view of this, a pyramidal feature fusion module is proposed in this paper to integrate the low-resolution, semantically strong features with high-resolution, semantically weak features. Then, an asymmetric Transformer structure is applied to predict the weights of the model. Finally, a feature refinement module is employed to optimize the search features. Extensive experiments on 5 mainstream datasets demonstrate the superiority of our tracker, where it has achieved better feature expression and more precise target localization.

Motion-aware object tracking for aerial images with deep features and discriminative correlation filter

AbstractObject tracking is a challenging task which is required for different problems such as surveillance, traffic analysis and human-computer interaction. The problem of tracking an object can be considered in different categories such as single object tracking, multiple object tracking, short-term tracking, long-term tracking, tracking by detection and detection-free tracking. This study focuses on detection-free tracking for ground targets on aerial images. The studies in the literature show that correlation filter and deep learning based object trackers perform well recently. This paper proposes a new correlation filter-based tracker containing a strategy for re-detection issue. We improve the performance of correlation filter-based tracker by adding a lightweight re-detection ability to the correlation filter tracker in case of a long occlusion or complete loss of target. We use deep features to train Discriminative Correlation Filter(DCF) by integrating sub-networks from pre-trained ResNet and SAND models. The experimental results on the popular UAV123L dataset show that the proposed method(MADCF) improves the performance of DCF tracker and have a reasonable performance for long-term tracking problem. Moreover, we prepare a new tracking dataset (PESMOD tracking) consisting of UAV images, and we evaluate the proposed method and state-of-the-art method in this dataset. We observed that the proposed method performs much better in ground target tracking from VIVID and PESMOD aerial datasets. The proposed MADCF tracker performs better for small targets tracked by UAVs compared to the deep learning-based trackers. The source code and prepared dataset are available at http://github.com/mribrahim/MADCF

Part-based Tracking via Discriminative Correlation Filters

In order to better deal with the partial occlusion issue, part-based trackers are widely used in visual object tracking recently. However, it is still difficult to realize fast and robust tracking, due to complicated online training and updating process. Correlation filters have been used in visual object tracking tasks recently because of their high efficiency. However, the traditional correlation filter based tracking methods do not deal with occlusion well. In this paper, we propose a novel tracking method which tracks objects based on parts with multiple correlation filters. The Bayesian inference framework and a structural constraint mask are adopted to enable our tracker to be robust to various appearance changes. Additionally, a discriminative part selection scheme is adopted to further improve performance and accelerate our method. Experimental results demonstrate that our multiple part tracker can significantly improve tracking performance on benchmark datasets.

Spatial Reliability Enhanced Correlation Filter: An Efficient Approach for Real-Time UAV Tracking

Traditional discriminative correlation filter (DCF) has received great popularity due to its high computational efficiency. However, the lightweight framework of DCF cannot promise robust performance when the tracker faces appearance variations within the background. These unpredictable appearance variations always distract the filter. Most existing DCF-based trackers either utilize deep convolutional features or incorporate additional constraints to elevate tracking robustness. Despite some improvements, both of them hamper the tracking speed and can only roughly alleviate the distractions of appearance variations. In this paper, a novel spatial reliability enhanced learning strategy is proposed to handle the problems aforementioned. By monitoring the variation of response produced in detection phase, a dynamic reliability map is generated to indicate the reliability of each background subregion. Then, label adjustment is conducted to repress the distractions of these unreliable areas. Compared with the conventional way of constraint where a new term is always added to realize the desired goal, label adjustment is simultaneously more efficient and effective. Moreover, to promise the accuracy and dependability of the reliability map, an adaptively updated response pool recording reliable historical response values is proposed. Extensive and exhaustive experiments on three challenging unmanned aerial vehicle (UAV) benchmarks, i.e., UAV123@10fps, DTB70 and UAVDT, which totally include 243 video sequences, validate the superiority of the proposed method against other state-of-the-art trackers and exhibit a remarkable generality in a variety of scenarios. Meanwhile, the tracking speed of 65.2FPS on a cheap CPU makes it suitable for real-time UAV applications.

SOCF: A correlation filter for real-time UAV tracking based on spatial disturbance suppression and object saliency-aware

The discriminative correlation filter (DCF) is commonly used in aerial object tracking due to its high tracking accuracy and computing speed. However, when similar object disturbances emerge in the background, the response map will generate sub-peaks, which may eventually lead to tracking failure. Meanwhile, the lack of attention to the tracked object can also cause tracking performance degradation. To these concerns, this paper proposes a novel correlation filter algorithm for real-time aerial tracking based on spatial disturbance suppression and object saliency-aware, i.e., SOCF. Firstly, this paper designs a novel spatial disturbance suppression strategy. Using the temporal information in the historical response maps, we construct a context response map, deviating it from the current response map to detect disturbance information in the background. Then, construct a spatial interference map, divide it into n×n non-overlapping regions, and suppress the negative samples in the disturbance region within the main regression. Furthermore, an object saliency-aware strategy is proposed, using a saliency detection algorithm to calculate the object-aware mask and multiplying it with the detection filter to obtain the object-aware filter. By constructing the object-aware regularization in the training phase, the trained detection filter focuses more on the object itself and can effectively separate the object from the background. Extensive experiments on four widely used unmanned aerial vehicle (UAV) datasets demonstrate that the proposed SOCF tracker achieves high tracking performance. Meanwhile, our tracker can maintain real-time aerial tracking at 48 FPS on a single CPU.

Learning Context Restrained Correlation Tracking Filters via Adversarial Negative Instance Generation.

The tracking performance of discriminative correlation filters (DCFs) is often subject to unwanted boundary effects. Many attempts have already been made to address the above issue by enlarging searching regions over the last years. However, introducing excessive background information makes the discriminative filter prone to learn from the surrounding context rather than the target. In this article, we propose a novel context restrained correlation tracking filter (CRCTF) that can effectively suppress background interference via incorporating high-quality adversarial generative negative instances. Concretely, we first construct an adversarial context generation network to simulate the central target area with surrounding background information at the initial frame. Then, we suggest a coarse background estimation network to accelerate the background generation in subsequent frames. By introducing a suppression convolution term, we utilize generative background patches to reformulate the original ridge regression objective through circulant property of correlation and a cropping operator. Finally, our tracking filter is efficiently solved by the alternating direction method of multipliers (ADMM). CRCTF demonstrates the accuracy performance on par with several well-established and highly optimized baselines on multiple challenging tracking datasets, verifying the effectiveness of our proposed approach.

Scale-Aware Tracking Method with Appearance Feature Filtering and Inter-Frame Continuity.

Visual object tracking is a fundamental task in computer vision that requires estimating the position and scale of a target object in a video sequence. However, scale variation is a difficult challenge that affects the performance and robustness of many trackers, especially those based on the discriminative correlation filter (DCF). Existing scale estimation methods based on multi-scale features are computationally expensive and degrade the real-time performance of the DCF-based tracker, especially in scenarios with restricted computing power. In this paper, we propose a practical and efficient solution that can handle scale changes without using multi-scale features and can be combined with any DCF-based tracker as a plug-in module. We use color name (CN) features and a salient feature to reduce the target appearance model's dimensionality. We then estimate the target scale based on a Gaussian distribution model and introduce global and local scale consistency assumptions to restore the target's scale. We fuse the tracking results with the DCF-based tracker to obtain the new position and scale of the target. We evaluate our method on the benchmark dataset Temple Color 128 and compare it with some popular trackers. Our method achieves competitive accuracy and robustness while significantly reducing the computational cost.

Tracking by segmentation with future motion estimation applied to person-following robots.

Person-following is a crucial capability for service robots, and the employment of vision technology is a leading trend in building environmental understanding. While most existing methodologies rely on a tracking-by-detection strategy, which necessitates extensive datasets for training and yet remains susceptible to environmental noise, we propose a novel approach: real-time tracking-by-segmentation with a future motion estimation framework. This framework facilitates pixel-level tracking of a target individual and predicts their future motion. Our strategy leverages a single-shot segmentation tracking neural network for precise foreground segmentation to track the target, overcoming the limitations of using a rectangular region of interest (ROI). Here we clarify that, while the ROI provides a broad context, the segmentation within this bounding box offers a detailed and more accurate position of the human subject. To further improve our approach, a classification-lock pre-trained layer is utilized to form a constraint that curbs feature outliers originating from the person being tracked. A discriminative correlation filter estimates the potential target region in the scene to prevent foreground misrecognition, while a motion estimation neural network anticipates the target's future motion for use in the control module. We validated our proposed methodology using the VOT, LaSot, YouTube-VOS, and Davis tracking datasets, demonstrating its effectiveness. Notably, our framework supports long-term person-following tasks in indoor environments, showing promise for practical implementation in service robots.

Learning dynamic spatial-temporal regularized correlation filter tracking with response deviation suppression via multi-feature fusion

Visual object tracking (VOT) for intelligent video surveillance has attracted great attention in the current research community, thanks to advances in computer vision and camera technology. Meanwhile, discriminative correlation filter (DCF) trackers garnered significant interest owing to their high accuracy and low computing cost. Many researchers have introduced spatial and temporal regularization into the DCF framework to achieve a more robust appearance model and further improve tracking performance. However, these algorithms typically set fixed spatial and temporal regularization parameters, which limit flexibility and adaptability under cluttered and challenging scenarios. To overcome these problems, in this work, we propose a new dynamic spatial–temporal regularization for the DCF tracking model that emphasizes the filter to concentrate on more reliable regions during the training stage. Furthermore, we present a response deviation-suppressed regularization term for responses to encourage temporal consistency and avoid model degradation by suppressing relative response changes between two consecutive frames. Moreover, we introduce a multi-memory tracking framework to exploit various features and each memory contributes to tracking the target across all frames. Significant experiments on the OTB-2013, OTB-2015, TC-128, UAV-123, UAVDT, and DTB-70 datasets have revealed that the performance thereof outperformed many state-of-the-art trackers based on DCF and deep-based frameworks in terms of tracking accuracy and tracking success rate.

GRM: Gaussian response module for visual tracking

Online Discriminative Correlation Filters have excellent performance in visual object tracking. It always divides the tracking network as a classification network and a regression network, which makes the regression network lack classification information and makes it hard to leverage its advantages. To address this problem, we propose the ATOM_GRM tracker to append the classification information to the network. Specifically, we design static and dynamic Gaussian response modules to encode the raw target position from classifier to regression network by multistage. In Particular, the parameters and fusion method of two Gaussian response modules is designed in different ways according to their location in the network. Moreover, we propose the ratio of height and width of the bounding box instead of intersection of union as the predictor, and two auxiliary training heads are used in the regression network. It makes the regression network better distinguish bounding boxes. Extensive experiments conducted on four public benchmarks, i.e., OTB100, GOT10k, LaSOT, Trackingnet, demonstrate the effectiveness of the proposed method. The proposed ATOM_GRM achieves 0.556 → 0.596 AO compared with the baseline ATOM on GOT10k.

A Re-detection Target Tracking Framework after Loss for Unmanned Aerial Vehicle

Currently, the target tracking algorithm based on discriminative correlation filter (DCF) and deep learning (DL) plays an increasingly important role in UAV (unmanned aerial vehicle). However, existing algorithms have limitations such as limited search region, difficulty in re-capturing targets when tracking is lost, and high computational complexity, which makes them difficult to apply to the UAV platform. In this paper, a target re-detection tracking algorithm (RDT) after tracking loss is designed for mobile platforms. RDT uses DCF in simple scenes to track targets and establish a target model between different frames. To ensure the algorithm when tracking is lost due to occlusion or moving out of view, an efficient switching criterion is designed to indirectly invoke the detection algorithm to re-capture the target and guide online learning of DCF. And the experiment results on the OTB benchmark show that RDT can re-capture the target after tracking is lost, and the speed of operation on the CPU is 85fps.

Robust Visual Object Tracking Based on Feature Channel Weighting and Game Theory

Although the discriminative correlation filter- (DCF)-based tracker improves tracking performance, some object representation issues can still be further optimized. On the one hand, the DCF tracker’s deep convolutional features contain many noisy channels, and assigning the same weights to multiple channels cannot distinguish the importance of different channels. On the other hand, a simple weighted fusion approach cannot fully utilize the benefits of different feature types. We propose a visual object tracking algorithm based on adaptive channel weighting and feature game fusion to solve these problems. In this study, an adaptive channel weighting strategy is designed to assign suitable weights to each channel based on the average energy ratio of the target and background regions in the feature channels and prune the channels with low weights to improve feature robustness and reduce computational complexity. Simultaneously, the game theory concept is introduced in the multifeature fusion. The handcrafted features are combined with shallow and deep convolutional features according to feature complementarity. Then, the two combined features are seen as two sides of the game, continuously gamed during the tracking process to generate a feature model with a higher representation capacity. Extensive experiments are conducted on four mainstream visual tracking benchmark datasets, including OTB2015, VOT2018, LaSOT, and UAV123. The experimental results show that the proposed algorithm performs outstandingly compared to the state-of-the-art trackers.

Pedestrian Tracking in UAV Images With Kalman Filter Motion Estimator and Correlation Filter

Visual object tracking is one of the significant parts of systems varying from autonomous driving to drone-based surveillance/tracking. Discriminative correlation filter (DCF) based trackers have proved their prominence in the past few years. These utilize visual information present in images for monitoring. This work uses the Kalman Filter (KF) to derive a motion estimation model. It combines it with two DCF-based trackers, namely Kernelized Correlation Filter (KCF) and Discriminative Correlation Filter with Channel and Spatial Reliability (CSRDCF) for pedestrian tracking. Camera motion-compensated versions of the trackers are also presented. The performance of the proposed methodology is presented in terms of Success Rate (SR) and Precision (P). Real-time power consumption, memory occupancy, and speed of trackers on the Jetson Nano (ARM Cortex-A57, 4 GB RAM) board have been presented.

Learning Background-Suppressed Dual-Regression Correlation Filters for Visual Tracking

The discriminative correlation filter (DCF)-based tracking method has shown good accuracy and efficiency in visual tracking. However, the periodic assumption of sample space causes unwanted boundary effects, restricting the tracker’s ability to distinguish between the target and background. Additionally, in the real tracking environment, interference factors such as occlusion, background clutter, and illumination changes cause response aberration and, thus, tracking failure. To address these issues, this work proposed a novel tracking method named the background-suppressed dual-regression correlation filter (BSDCF) for visual tracking. First, we utilize the background-suppressed function to crop out the target features from the global features. In the training step, while introducing the spatial regularity constraint and background response suppression regularization, we construct a dual regression structure to train the target and global filters separately. The aim is to exploit the difference between the output response maps for mutual constraint to highlight the target and suppress the background interference. Furthermore, in the detection step, the global response can be enhanced by a weighted fusion of the target response to further improve the tracking performance in complex scenes. Finally, extensive experiments are conducted on three public benchmarks (including OTB100, TC128, and UAVDT), and the experimental results indicate that the proposed BSDCF tracker achieves tracking performance comparable to many state-of-the-art (SOTA) trackers in a variety of complex situations.

RGBT tracking based on prior least absolute shrinkage and selection operator and quality aware fusion of deep and handcrafted features

Object tracking has always been one of the most challenging topics in machine vision. In recent years, trackers have used RGB-T datasets to overcome the limitations of single-modality data. This paper introduces a Prior Least Absolute Shrinkage and Selection Operator (PLASSO)-based cost function optimized by the Alternating Direction Method of Multipliers (ADMM) for a Discriminative Correlation Filter (DCF)-based tracker. Detailed closed-form solutions for these optimization problems are also given in this paper. The proposed PLASSO-based approach adaptively determines the scale, parameters, and features of the PLASSO-ADSPF tracker. PLASSO- ADSPF first finds the response map in the Fourier domain for the input images and then uses an efficient adaptive function to fuse the results. Based on the response maps, the proposed tracker adaptively and consciously extracts features (handcraft and Deep Features (DF)) to improve the speed of the tracking algorithm. PLASSO-ADSPF extracts DF from the convolution layer output of a modified pre-trained deep network via SoftMax Tsallis Entropy (TsEn) as a proposed cost function in the last layer. Also, a graph is proposed to determine longitudinal and transverse scales independently and accurately. Moreover, a new bounding box regression is proposed based on PLASSO and Cholesky factorization to refine the target box. Extensive experiments have been performed for the PLASSO-ADSPF tracker on five popular datasets. The results show the superiority of the proposed approach compared to the state-of-the-art trackers. Our code is available at PLASSO-ADSPF-tracker.

Learning spatio-temporal discriminative model for affine subspace based visual object tracking

Discriminative correlation filters (DCF) with powerful feature descriptors have proven to be very effective for advanced visual object tracking approaches. However, due to the fixed capacity in achieving discriminative learning, existing DCF trackers perform the filter training on a single template extracted by convolutional neural networks (CNN) or hand-crafted descriptors. Such single template learning cannot provide powerful discriminative filters with guaranteed validity under appearance variation. To pinpoint the structural relevance of spatio-temporal appearance to the filtering system, we propose a new tracking algorithm that incorporates the construction of the Grassmannian manifold learning in the DCF formulation. Our method constructs the model appearance within an online updated affine subspace. It enables joint discriminative learning in the origin and basis of the subspace, achieving enhanced discrimination and interpretability of the learned filters. In addition, to improve tracking efficiency, we adaptively integrate online incremental learning to update the obtained manifold. To this end, specific spatio-temporal appearance patterns are dynamically learned during tracking, highlighting relevant variations and alleviating the performance degrading impact of less discriminative representations from a single template. The experimental results obtained on several well-known datasets, i.e., OTB2013, OTB2015, UAV123, and VOT2018, demonstrate the merits of the proposed method and its superiority over the state-of-the-art trackers.

Intersection analysis using computer vision techniques with SUMO

Abstract This paper presents intersection analysis using computer vision techniques with Simulation of Urban MObility (SUMO). First, an efficient deep-visual tracking pipeline is proposed by using the off-the-shelf YOLO object detection architecture and cascading it with a discriminative correlation filter to produce reliable trajectories for traffic analysis of vehicles and pedestrians. While a variety of traffic measurements can be directly estimated from the extracted trajectories (e.g., speed, turning movement count), a method of incorporating turning movement count (TMC) within SUMO is proposed in order to mimic a realistic traffic flow for an observed intersection and its comprehensive analysis. Experimental evaluations on the developed tracking system implies that the YOLOv5 variant is the best for traffic cameras and, after appropriate fine-tuning using the University of Nevada, Las Vegas pedestrian data set, the YOLOv5 performance manifested a significant improvement with a recall value of 0.62. The tracking system is further employed for monitoring three other intersections in the downtown area of Las Vegas and turning movement counts were estimated for peak hours in the morning and evening of one day (7:00–9:00 and 16:00–18:00) at 15-min intervals. Finally, the intersection design, including traffic signals with estimated TMC, is used to calibrate SUMO to provide critical parameters (e.g., lane density, travel time, occupancy) for traffic signal performance evaluation and comprehensive intersection analysis. The signal design treatment demonstrates a significant improvement in travel times and simulation results indicate that the turning-left ratio is a crucial factor affecting the travel time of vehicles on each intersection leg.

Dynamic Learning Rate of Template Update for Visual Target Tracking

The trackers based on discriminative correlation filter (DCF) have achieved remarkable performance in visual target tracking in recent years. Since the targets are usually affected by various factors such as deformation, rotation, motion blur and so on, the trackers have to update the templates for tracking online. The purpose of template update is to adapt to the target changes, the magnitude of which is closely related to the motion state of the target. Actually, the learning rate of template update indicates the weight of the historical sample, and its value is fixed in most existing trackers, which will decrease the precision of the tracker or make the tracker unstable. In this study, a new dynamic learning rate method for template update is proposed for visual target tracking. The motion state of the target is defined by the difference in target center position between the frames. Then, the learning rate is adjusted dynamically according to the motion state of the target instead of the fixed value, which could achieve better performance. Experiments on the popular datasets OTB100 and UAV123 show that with the proposed dynamic learning rate for template update, the DCF-based trackers can improve tracking accuracy and obtain better tracking stability in scenarios such as fast movement and motion blur.