Correlation Filter Tracking Research Articles

Robust visual tracker integrating adaptively foreground segmentation into multi-feature fusion framework

Existing Discriminative Correlation Filter (DCF) based methods suffer from the limitations of rectangular shape assumptions. Aiming at this issue, in this paper, we propose an effective tracking approach which integrates a pixel-wise foreground segmentation mask into the correlation filter within a multi-feature fusion framework. Specifically, we first propose a novel segmentation algorithm which combines the color histogram with the spatial prior. On this basis, we implement a target-masked correlation filter (TMCF) tracker by introducing the foreground mask into a ridge regression, which successfully suppresses unexpected background information inside the bounding box. Secondly, we apply the alternating direction method of multipliers (ADMM) to solve our TMCF model efficiently to obtain the closed-form solution. Finally, a complementary fusion tracker by the combining of TMCF and color histogram scores (fTMCFCH) is formulated, which is robust to deformations and illumination changes simultaneously. The fusion factor is determined adaptively by the reliability derived from the target resolution of the trackers separately in each frame. We perform extensive experiments on three benchmarks: OTB-2013, OTB-2015 and Temple-Color-128. The concrete experimental results demonstrate that our tracker outperforms several state-of-the-art trackers.

A Visual Tracking Method Based on an Adaptive Overlapping Correlation Filter for Robotic Real-Time Cognitive Imaging

Computer vision is a very important research direction in the cognitive computing field. Robots encounter various target-tracking problems with computer vision systems. Robust scale estimation is an important issue in tracking algorithms. Most of the available methods have difficulty addressing even reasonable changes of scale in complex videos. In this paper, we propose a visual tracking method based on robust scale estimation, which uses a discriminant correlation filter based on a time-dependent scale-space filter and an adaptive cross-correlation filter. The tracker uses separate essential filters for sample migration and scale estimation. Furthermore, the built-in scale estimation method can be introduced into other tracking algorithms. We validate the proposed method on the UAV123 dataset. The results of comparison experiments with the traditional correlation filter tracking method demonstrate that the proposed method improves the success rate and tracking accuracy while controlling the computational complexity; its success rate measured by the area under the curve is 0.638, while at a location error precision of 20%, it is 0.649.

Adaptive multi-branch correlation filters for robust visual tracking

In recent years, deep convolutional features have been applied to discriminative correlation filters-based methods, which have achieved impressive performance in tracking. Most of them utilize hierarchical features from a certain layer. However, this is not always sufficient to learn target appearance changes and to suppress the background interference in complicated interfering factors (e.g., deformation, fast motion, low resolution, and rotations). In this paper, we propose an adaptive multi-branch correlation filter tracking method, by constructing multi-branch models and using an adaptive selection strategy to improve the accuracy and robustness of visual tracking. Specially, the multi-branch models are introduced to tolerate temporal changes of the object, which can serve different circumstances. In addition, the adaptive selection strategy incorporates both foreground and background information to learn background suppression. To further improve the tracking performance, we propose a measurement method to handle tracking failures from unreliable samples. Extensive experiments on OTB-2013, OTB-2015, and VOT-2016 datasets show that the proposed tracker has comparable performance compared to state-of-the-art tracking methods. Especially, on the OTB-2015, our method significantly improves the baseline with a gain of 5.5% in overlap precision.

Location-Aware and Regularization-Adaptive Correlation Filters for Robust Visual Tracking.

Correlation filter (CF) has recently been widely used for visual tracking. The estimation of the search window and the filter-learning strategies is the key component of the CF trackers. Nevertheless, prevalent CF models separately address these issues in heuristic manners. The commonly used CF models directly set the estimated location in the previous frame as the search center for the current one. Moreover, these models usually rely on simple and fixed regularization for filter learning, and thus, their performance is compromised by the search window size and optimization heuristics. To break these limits, this article proposes a location-aware and regularization-adaptive CF (LRCF) for robust visual tracking. LRCF establishes a novel bilevel optimization model to address simultaneously the location-estimation and filter-training problems. We prove that our bilevel formulation can successfully obtain a globally converged CF and the corresponding object location in a collaborative manner. Moreover, based on the LRCF framework, we design two trackers named LRCF-S and LRCF-SA and a series of comparisons to prove the flexibility and effectiveness of the LRCF framework. Extensive experiments on different challenging benchmark data sets demonstrate that our LRCF trackers perform favorably against the state-of-the-art methods in practice.

Real-Time Visual Tracking of Moving Targets Using a Low-Cost Unmanned Aerial Vehicle with a 3-Axis Stabilized Gimbal System

Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping, which are widely used in reconnaissance, surveillance, and target acquisition (RSTA) applications. In this paper, we present an onboard vision-based system for low-cost UAVs to autonomously track a moving target. Real-time visual tracking is achieved by using an object detection algorithm based on the Kernelized Correlation Filter (KCF) tracker. A 3-axis gimbaled camera with separate Inertial Measurement Unit (IMU) is used to aim at the selected target during flights. The flight control algorithm for tracking tasks is implemented on a customized quadrotor equipped with an onboard computer and a microcontroller. The proposed system is experimentally validated by successfully chasing a ground and aerial target in an outdoor environment, which has proven its reliability and efficiency.

Correlation filters based on temporal regularization and background awareness,

Information about background appearance and previous tracked frames is significant for effectively discriminating a target from a complex scene. In this paper, we propose a novel tracker called temporal regularization and background-aware correlation filter (TRBACF) tracker based on the theory of correlation filter. In the proposed TRBACF tracker, an improved circular shift operation for collecting training samples is used to obtain more background information, which can enhance the discrimination ability of the learned correlation filters. Furthermore, to ensure the long-term tracking performance, a temporal regularization term is added to the appearance model of the classical correlation filter. The developed appearance model can take advantage of the similarity of the filters in the adjacent frames and improve the learned filter to be more adaptive to variations in the scene. Extensive experimental results on various challenging videos demonstrate that the proposed TRBACF tracker achieves superior accuracy than some state-of-the-art trackers.

Scale-adaptive Correlation Filter Tracking Algorithm Combined with Statistical Color Histogram Feature

The algorithm based on kernel correlation filtering shows excellent performance. Aiming at the problem that traditional correlation filtering algorithms are sensitive to deformation and cannot solve the tracking failure caused by scale changes, this paper proposes a scale-adaptive correlation filtering algorithm combining statistical color feature. Firstly, predict and train the HOG features of the correlation filter and the color histogram features to obtain the respective response values, namely the template response and the histogram response. Then adopt the block method, and perform scale prediction based on the position of each response point. Finally, the final fusion response is calculated. The experimental results demonstrate that compared with traditional KCF, when the target scale has large scale change and deformation, the improved algorithm in this paper performs better, and can still track stably.

Noise-Aware Framework for Robust Visual Tracking.

Both siamese network and correlation filter (CF)-based trackers have exhibited superior performance by formulating tracking as a similarity measure problem, where a similarity map is learned by the correlation between a target template and a region of interest (ROI) with a cosine window. Nevertheless, this window function is usually fixed for various targets and not changed, undergoing significant noise variations during tracking, which easily makes model drift. In this article, we focus on the study of a noise-aware (NA) framework for robust visual tracking. To this end, the impact of various window functions is first investigated in visual tracking. We identify that the low signal-to-noise ratio (SNR) of windowed ROIs makes the above trackers degenerate. At the prediction phase, a novel NA window customized for visual tracking is introduced to improve the SNR of windowed ROIs by adaptively suppressing the variable noise according to the observation of similarity maps. In addition, to further optimize the SNR of windowed pyramid ROIs for scale estimation, we propose to use the particle filter to dynamically sample several windowed ROIs with more favorable signals in temporal domains instead of this pyramid ROIs extracted in spatial domains. Extensive experiments on the popular OTB-2013, OTB-50, OTB-2015, VOT2017, TC128, UAV123, UAV123@10fps, UAV20L, and LaSOT datasets show that our NA framework can be extended to many siamese and CF trackers and our variants obtain superior performance than baseline trackers with a modest impact on efficiency.

Scale-Adaptive Context-Aware Correlation Filter with Output Constraints for Visual Target Tracking

Context-aware correlation filter tracker is one of the most advanced target trackers, and it has significant improvement in tracking accuracy and success rate compared with traditional trackers. However, because the complexity of background in the process of tracking can lead to inaccurate output response of target tracking, an accurate tracking model is difficult to be established. Moreover, the drift problem is easy to occur during the tracking process due to the imprecise tracking model, especially when the target has large area occlusion, fast motion, and deformation. Aiming at the drift problem in the target tracking process, a novel algorithm is proposed in this paper. The developed method derives the specific representation of constraint output by assuming that the output response is Gaussian distribution, and a variable update parameter is obtained based on the output constraint relationship at first, then the tracking filter is selectively updated with changeable update parameters and fixed update parameters, and finally, the target scale is updated with maximizing posterior probability distribution. The effectiveness of developed algorithm is verified by comparing with other trackers on OTB-50 and OTB-100 evaluation benchmark datasets, and the experimental results have shown that the suggested tracker has higher overall object tracking performance than other trackers.

Collaborative model tracking with robust occlusion handling

Currently, the discriminative correlation filter-based trackers have achieved higher tracking accuracy. However, visual tracking still faces challenges in terms of heavy occlusion, scale variation and so on. In this study, the authors intend to solve heavy occlusion by introducing collaborative model into classifier-box. Firstly, they introduce complex colour features into correlation filter tracker to improve the effect of the tracker. Secondly, they introduce a multi-scale method into their tracker to ease the scale problem. Thirdly, in order to solve the heavy occlusion in the tracking process, they adopt the locally weighted distance and classifier-box. Their algorithm achieves distance precision rates of 81.7 and 77.4% on OTB2013 dataset and OTB2015 dataset, respectively. Their contribution focuses on solving heavy occlusion by using colour features, locally weighted distance and classifier-box. The experimental results on OTB2013 and OTB2015 datasets demonstrate their algorithm to perform better than state-of-the-art methods.

High Confidence Tracking with Offline Historical Learning and Online Correlation Filter Updating

Target tracking is one of the most challenging tasks in computer vision. In this paper, the high confidence tracking (HCT) algorithm is proposed by combining the offline historical learning network with online correlation filter updating model. First, the weighted historical targets are introduced into the offline learning network, which solves the problem of target loss caused by inaccurate tracking of the previous frame. Second, the target’s confidence detection mechanism is proposed, and added to the correlation filter tracking algorithm, so that the model drift is avoided. Finally, we form a new high confidence tracking algorithm with offline learning. Compared with the state-of-the-art tracking algorithm, our algorithm performs outstandingly on benchmark OTB13 and OTB15, while ensuring real-time performance.

The left-behind human detection and tracking system based on vision with multi-model fusion and microwave radar inside the bus

Left-behind humans inside the car or bus have caused a lot of accidents, so it is essential to detect the humans in vehicle. Current human detection methods rely on wearable devices, oxygen sensors, and special seat designs in vehicles, but those sensors cannot adapt to ever-changing environments. To solve those problems and especially to improve passengers’ safety on the bus, we propose a method to accomplishing human detection by fusion vision and microwave radar information in various environments in vehicle. For vision information, we use different networks to extract human and human face features, and fusion of the detection results in different models to improve human detection accuracy. The human detection model is MobileNet-V2, and the human face detection model is MTCNN. A new matching schedule and tracking objects management rule based on the Kernelized Correlation Filter tracker are designed to track the human and human face detection boxes. The microwave radar information is used to detect moving objects. Finally, the fusion vision and microwave radar detection results are implemented. Experiments show that our method has improved the human detection accuracy in vehicle, and this method can be used for detection of left-behind children on the school bus.

Visual object tracking using sparse context-aware spatio-temporal correlation filter

This paper presents a novel sparse context-aware spatio-temporal correlation filter tracker (SCAST) method for robust visual object tracking. Different from the existing trackers, this paper introduce an l1 multi-scale regularization parameter-based correlation filter that reduces the boundary effect due to partial occlusions, illumination and scale variations. At each iteration, the l1 regularization parameter is updated through spatial knowledge of each correlation filter coefficient. Besides, the contextual information acquired from the target region can lead to determining the accurate localization of the target. Moreover, contextual information has combined with spatio-temporal factor to achieve the better performance. Further, an objective function is designed with system constraints to ensure the applicability of the model and the optimal solution is derived by utilizing the alternating direction method of multiplier, which leads to low computational cost. Finally, the feasibility and superiority of proposed tracker algorithm is evaluated through three benchmark dataset: OTB-2013, OTB-2015, and TempleColor-128.

A vehicle tracking algorithm combining detector and tracker

Real-time multichannel video analysis is significant for intelligent transportation. Considering that deep learning and correlation filter (CF) tracking are time-consuming, a vehicle tracking method for traffic scenes is presented based on a detection-based tracking (DBT) framework. To design the model of vehicle detection, the You Only Look Once (YOLO) model is used, and then, two constraints including object attribute information and intersection over union (IOU), are combined to modify the vehicle detection box. This approach improves vehicle detection precision. In the design of tracking model, a lightweight feature extraction network model for vehicle tracking is constructed. An inception module is used in this model to reduce the computational load and increase the adaptivity of the network scale. And a squeeze-and-excitation channel attention mechanism is adopted to enhance feature learning. Regarding the object tracking strategy, the method of combining a spatial constraint and filter template matching is adopted. The observation value and prediction value are matched and corrected to achieve stable tracking of the target. Based on the interference of occlusion in target tracking, the spatial position, moving direction, and historical feature correlation of the target are comprehensively employed to achieve continuous tracking of the target.

Robust fusion for RGB-D tracking using CNN features

Recently, RGB-D sensors have become popular. Many computer vision problems can be better dealt with depth data. It is a challenging problem to integrate depth data into a visual object tracker to address the problems such as scale change and occlusion. In this paper, we propose a robust fusion based RGB-D tracking method. Specifically, hierarchical convolutional neural network (CNN) features are first adopted to encode RGB and depth images separately. Next, target is tracked based on correlation filter tracking framework. Then the results of each CNN feature are localized according to the tracking results in a short period of time. Finally, the target is localized by jointly fusing the results of RGB and depth images. Model updating is finally carried out according to the differences between RGB and depth images. Experiments on the University of Birmingham RGB-D Tracking Benchmark (BTB) and the Princeton RGB-D Tracking Benchmark (PTB) achieve comparable results to state-of-the-art methods.

A target response adaptive correlation filter tracker with spatial attention

In recent years, many efficient and accurate algorithms have been proposed in the field of target tracking. The existing correlation filter(CF) based tracking expands the sample through cyclic shift and uses the position of the maximum response map as the most likely position of the target. Since the cyclic shift sample is not the real sample, the target response may also occur in the non-target region, which not only causes the boundary effect problem but also results in the positioning error. To train the filter with the wrong position will lead to the tracking drift. This paper proposes a target response adaptive correlation filter tracker with spatial attention to solve the above problems. Firstly, more useful feature information can be learned by making full use of the context information of the target area, and spatial attention mechanisms can be introduced to suppress the background information and reduce the unnecessary boundary effect. Secondly, the dynamic change of the target response is captured, and the most reliable response map is selected when the interference response map appears, to reduce the probability of dispositioning and train the more recognizable filter. Extensive experimental results on popular benchmarks(OTB2013 OTB100 and VOT2016) demonstrate that the proposed tracker performs favorably against other state-of-the-art tracking methods, with real-time performance.

Multi-part and scale adaptive visual tracker based on kernel correlation filter

Accurate visual tracking is a challenging issue in computer vision. Correlation filter (CF) based methods are sought in visual tracking based on their efficiency and high performance. Nonetheless, CF-based trackers are sensitive to partial occlusion, which may reduce their overall performance and even lead to failure in tracking challenge. In this paper, we presented a very powerful tracker based on the kernelized correlation filter tracker (KCF). Firstly, we employ an intelligent multi-part tracking algorithm to improve the overall capability of correlation filter based tracker, especially in partial-occlusion challenges. Secondly, to cope with the problem of scale variation, we employ an effective scale adaptive scheme, which divided the target into four patches and computed the scale factor by finding the maximum response position of each patch via kernelized correlation filter. With this method, the scale computation was transformed into locating the centers of the patches. Thirdly, because the small deviation of the central function value will bring the problem of location ambiguity. To solve this problem, the new Gaussian kernel functions are introduced in this paper. Experiments on the default 51 video sequences in Visual Tracker Benchmark demonstrate that our proposed tracker provides significant improvement compared with the state-of-art trackers.

Learning salient features to prevent model drift for correlation tracking

Correlation Filter (CF) based algorithms play an important role in the field of Visual Object Tracking (VOT) due to their high accuracy and low computational complexity. While existing CF tracking algorithms suffer performance degradation due to inaccurate object modeling. In this paper, we improve the object modeling accuracy in both CF training stage and target detection procedure to preventing the drift problem. Specifically, we propose a multi-model structure for CF trackers to capture the target appearance changes, where different appearance models are trained with specific samples to catch the salient features of the target and reduce the computational cost. Furthermore, a space filter for detection features is designed to suppress the boundary effect under Gaussian motion prior, which contributes to improving the accuracy of position estimation. We deploy our method to three hand-crafted features based CF trackers to perform real-time visual tracking on popular benchmarks. The experimental results demonstrate the efficacy of our proposed scheme and the efficiency of our trackers. In addition, we provide a comprehensive analysis of the proposed method to facilitate application.

Real-Time Perception and Positioning for Creature Picking of an Underwater Vehicle

This study addresses the ability of an underwater vehicle-manipulator system (UVMS) to perceive submarine creatures in the complex undersea environment. A real-time system for Submarine Creature Perception and Positioning (SCPP) based on a binocular vision system is proposed. In particular, we adopt a new Four-Corner Neighborhood Matching Method (FCNMM) to obtain specific target positions. A Fusion Correction Mechanism (FCM) is added to the Kernelized correlation filter (KCF) tracking algorithm to improve tracking performance. Furthermore, SCPP is applied in our UVMS platform to perform seafloor creature picking at the Zhangzidao seafood farm in Dalian, China. The experimental results show the feasibility and robustness of the SCPP system.

Research and implementation of multi-object tracking based on vision DSP

This paper applies a pragmatic approach to study the real-time performance effect of software design methods for multiple object tracking (MOT) based on vision digital signal processing (vision DSP). The MOT system in the paper combines target detection, the Hungarian algorithm and the Kernel correlation filter (KCF) tracker. In addition, the MOT system needs to support multiway video streams, so higher speed and storage requirements are necessary for target tracking. Therefore, we carried out some studies on how to improve tracker speed performance and reduce system resource consumption under limited system resources. In the paper, we achieved the goal in two respects. Regarding the data processing, we studied how to efficiently process tracking data by utilizing the parallel characteristics of iDMA (integrated direct memory access) and a DSP core; and regarding the data storage, we proposed a time-sharing strategy to solve the DSP local memory (data RAM) usage issue for multiple tracking objects. In addition, regarding the software design, we propose a new strategy, which includes two levels of parallel computations: the frame-level parallel computations and the tracking object-level parallel computations. The experimental results show that the KCF tracking algorithm based on vision DSP achieves not only the desired real-time tracking speed but also the expected goal of system resource utilization. Our research methods also provide a reference for algorithm embedded applications in the field of computer vision.