Kernelized Correlation Filter Tracker Research Articles

The left-behind human detection and tracking system based on vision with multi-model fusion and microwave radar inside the bus

Left-behind humans inside the car or bus have caused a lot of accidents, so it is essential to detect the humans in vehicle. Current human detection methods rely on wearable devices, oxygen sensors, and special seat designs in vehicles, but those sensors cannot adapt to ever-changing environments. To solve those problems and especially to improve passengers’ safety on the bus, we propose a method to accomplishing human detection by fusion vision and microwave radar information in various environments in vehicle. For vision information, we use different networks to extract human and human face features, and fusion of the detection results in different models to improve human detection accuracy. The human detection model is MobileNet-V2, and the human face detection model is MTCNN. A new matching schedule and tracking objects management rule based on the Kernelized Correlation Filter tracker are designed to track the human and human face detection boxes. The microwave radar information is used to detect moving objects. Finally, the fusion vision and microwave radar detection results are implemented. Experiments show that our method has improved the human detection accuracy in vehicle, and this method can be used for detection of left-behind children on the school bus.

Multi-part and scale adaptive visual tracker based on kernel correlation filter

Accurate visual tracking is a challenging issue in computer vision. Correlation filter (CF) based methods are sought in visual tracking based on their efficiency and high performance. Nonetheless, CF-based trackers are sensitive to partial occlusion, which may reduce their overall performance and even lead to failure in tracking challenge. In this paper, we presented a very powerful tracker based on the kernelized correlation filter tracker (KCF). Firstly, we employ an intelligent multi-part tracking algorithm to improve the overall capability of correlation filter based tracker, especially in partial-occlusion challenges. Secondly, to cope with the problem of scale variation, we employ an effective scale adaptive scheme, which divided the target into four patches and computed the scale factor by finding the maximum response position of each patch via kernelized correlation filter. With this method, the scale computation was transformed into locating the centers of the patches. Thirdly, because the small deviation of the central function value will bring the problem of location ambiguity. To solve this problem, the new Gaussian kernel functions are introduced in this paper. Experiments on the default 51 video sequences in Visual Tracker Benchmark demonstrate that our proposed tracker provides significant improvement compared with the state-of-art trackers.

Research and implementation of multi-object tracking based on vision DSP

This paper applies a pragmatic approach to study the real-time performance effect of software design methods for multiple object tracking (MOT) based on vision digital signal processing (vision DSP). The MOT system in the paper combines target detection, the Hungarian algorithm and the Kernel correlation filter (KCF) tracker. In addition, the MOT system needs to support multiway video streams, so higher speed and storage requirements are necessary for target tracking. Therefore, we carried out some studies on how to improve tracker speed performance and reduce system resource consumption under limited system resources. In the paper, we achieved the goal in two respects. Regarding the data processing, we studied how to efficiently process tracking data by utilizing the parallel characteristics of iDMA (integrated direct memory access) and a DSP core; and regarding the data storage, we proposed a time-sharing strategy to solve the DSP local memory (data RAM) usage issue for multiple tracking objects. In addition, regarding the software design, we propose a new strategy, which includes two levels of parallel computations: the frame-level parallel computations and the tracking object-level parallel computations. The experimental results show that the KCF tracking algorithm based on vision DSP achieves not only the desired real-time tracking speed but also the expected goal of system resource utilization. Our research methods also provide a reference for algorithm embedded applications in the field of computer vision.

A Novel Scale Insensitive KCF Tracker Based on HOG and Color Features

Kernelized Correlation Filters (KCF) for visual tracking have received much attention due to their fast speed and outstanding performances in real scenarios. However, the KCF sometimes still fails to track the targets with different scales, and it may drift because the target response is fixed and the original histogram of orientation gradient (HOG) features cannot represent the targets well. In this paper, we propose a novel fast tracker, which is based on KCF and insensitive to scale changes by learning two independent correlation filters (CFs) where one filter is designed for position estimation and the other is for scale estimation. In addition, it can adaptively change the target response and multiple features are integrated to improve the performance for our tracker. Finally, we employ an adaptive high confidence filters updating scheme to avoid errors. Evaluated on the popular OTB50 and OTB100 datasets, our proposed trackers show superior performances in terms of efficiency and accuracy compared to the existing methods.

Towards Real-Time Eyeblink Detection in the Wild: Dataset, Theory and Practices

Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing. Despite previous efforts, most of existing focus on addressing the eyeblink detection problem under constrained indoor conditions with relative consistent subject and environment setup. Nevertheless, towards practical applications, eyeblink detection in the wild is highly preferred, and of greater challenges. In this paper, we shed the light to this research topic. A labelled eyeblink in the wild dataset (i.e., HUST-LEBW) of 673 eyeblink video samples (i.e., 381 positives, and 292 negatives) is first established. These samples are captured from the unconstrained movies, with the dramatic variation on face attribute, head pose, illumination condition, imaging configuration, etc. Then, we formulate eyeblink detection task as a binary spatial-temporal pattern recognition problem. After locating and tracking human eyes using SeetaFace engine and KCF (Kernelized Correlation Filters) tracker respectively, a modified LSTM model able to capture the multi-scale temporal information is proposed to verify eyeblink. A feature extraction approach that reveals the appearance and motion characteristics simultaneously is also proposed. The experiments on HUST-LEBW reveal the superiority and efficiency of our approach. The comparisons with the existing state-of-the-art methods validate the advantages of our manner for eyeblink detection in the wild.

Can We Track Targets From Space? A Hybrid Kernel Correlation Filter Tracker for Satellite Video

Despite the great success of correlation filter-based trackers in visual tracking, it is questionable whether they can still perform on the satellite video data, acquired by a satellite or space station very high above the earth. The difficulty lies in that the targets usually occupy only a few pixels compared with the image size of over one million pixels and almost melt into the similar background. Since correlation filter models strongly depend on the quality of features and the spatial layout of the tracked object, they would probably fail on satellite video tracking tasks. In this paper, we propose a hybrid kernel correlation filter (HKCF) tracker employing two complementary features adaptively in a ridge regression framework. One feature is the optical flow that can detect variation pixels of the target. The other one is the histogram of oriented gradient that can capture the contour and texture information in the target, and an adaptive fusion strategy is proposed to employ the strengths of both features in different satellite videos. Quantitative evaluations are performed on six real satellite video data sets. The results show that our approach outperforms state-of-the-art tracking methods while running at more than 100 frames/s.

Scale Adaptive Target Tracking Algorithm Based on Correlation Filtering

Focusing on the issue that the kernel correlation filter tracking method can not accurately adapt to the target scale change, a scale estimation method is added under the kernel correlation filter tracking frame, and a scale adaptive target tracking algorithm is proposed. The algorithm firstly uses the kernel function to obtain the kernel correlation filter, and estimates the position of the target for the candidate sample detection; then uses the scale estimation method to detect the target scale to further determine the optimal scale of the target to be tracked in the current frame; Finally, according to the occlusion judgment mechanism, the parameters of the target performance model and the scale model are updated online. The experimental results show that compared with kernel correlation filtering algorithm, the proposed algorithm can accurately target tracking process to adapt to changes in the target scale, and the center position error reduced by an average of 4.39 pixels.

An improved Kernelized Correlation Filter tracking algorithm based on multi-channel memory model

Aiming at the problems of serious occlusions, deformations, background clutters and so on in the process of target tracking, an improved Kernelized Correlation Filter (KCF) tracking algorithm based on multi-channel memory model is proposed in this paper. Firstly, an updating model based on multi-channel memory is established, in which a control channel is used for memorizing target template, and two executive channels are used for memorizing the parameters and feature of classifier. Then, the established multi-channel memory model is introduced into the updating process of classifier. Our experimental results show that the proposed algorithm can achieve accurate and robust target tracking under the conditions of occlusions, deformations and background clutters.

Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion

In this paper, a tracker scheme is proposed that not only can face object tracking challenges but also can estimate object positions over occluded frames. In the proposed scheme, kernelized correlation filter (KCF) is considered as our basic tracker due to its high efficiency in the most object tracking challenges except occlusion and illumination variation. To improve the efficiency of KCF, the proposed method integrates an occlusion detection method, an adaptive model update and a prediction system into the KCF tracker. The occlusion detection method is based on the peak-to-sidelobe ratio of the confidence map to determine the type of occlusion. When an object is partially occluded, the object appearance model is adaptively updated to increase the accuracy of object tracking. When full occlusion occurs, the proposed predictor is run and exploits the available motion information before the occurrence of full occlusion to predict the location of the tracked object. The proposed predictor uses adaptive transition state equations to estimate the acceleration and velocity of the object needed in the extended Kalman filter (EKF) to predict object position. It also uses two quadratic equations to estimate the object trajectory. Finally, a method is proposed that exploits the estimated object positions by both EKF and the object trajectory to predict object positions over fully occluded frames. Experimental results on open datasets show that the proposed method achieved a better performance in comparison with several state-of-the-art trackers.

Airborne visual tracking and reidentification system

Visual tracking and reidentification has gathered a lot of attention in real-world surveillance applications. We explore on-board visual tracking and reidentification of a person from a moving camera, such as a drone. We develop an android application which can communicate with the drone and transmit roll, pitch, and vertical upthrust instructions. We use a kernelized correlation filter tracker for tracking the person, and for the reidentification task we apply a deep three-dimensional ConvNet. The experiments demonstrate a robust performance.

Adaptive Model Update Strategy for Correlation Filter Trackers

Model update is an important module in target trackers. It plays an important role in adaptive tracking. Many researches have proven that different model update strategies should be adopted, when tracking in different scenes, especially in occlusion and deformation. Though many strategies have been proposed in recent years, few of them make high improvement and good combination on trackers. In this paper, we first proved there is a close relationship between the tracking scenes and the response maps. Then, we proposed an adaptive model update strategy for calculating model update rate based on the response map. Many experiments have been done to compare the proposed model update strategy with some state-of-the-art strategies, and the results have shown that the proposed model update strategy outperforms the best model update strategy by 7% on the test of Kernel Correlation Filter tracker. Furthermore, the proposed model update strategy was evaluated on some state-of-the-art correlation filter trackers. Results have shown the proposed strategy was well integrated into many trackers, and improved the tracking accuracy effectively.

A template consensus method for visual tracking

Visual tracking is a challenging problem in computer vision. Recently, correlation filter-based trackers have shown to provide excellent tracking performance. Inspired by a sample consensus approach proposed for foreground detection, which classifies a given pixel as foreground or background based on its similarity to recently observed samples, we present a template consensus tracker based on the kernelized correlation filter (KCF). Instead of keeping only one target appearance model in the KCF, we make a feature pool to keep several target appearance models in our method and predict the new target position by searching for the location of the maximal value of the response maps. Both quantitative and qualitative evaluations are performed on the CVPR2013 tracking benchmark dataset. The results show that our proposed method improves the original KCF tracker by 8.17% in the success plot and 8.11% in the precision plot.

Particle filter re-detection for visual tracking via correlation filters

Most of the correlation filter based tracking algorithms can achieve good performance and maintain fast computational speed. However, in some complicated tracking scenes, there is a fatal defect that causes the object to be located inaccurately. In order to address this problem, we propose a particle filter redetection based tracking approach for accurate object localization. During the tracking process, the kernelized correlation filter (KCF) based tracker locates the object by relying on the maximum response value of the response map; when the response map becomes ambiguous, the KCF tracking result becomes unreliable. Our method can provide more candidates by particle resampling to detect the object accordingly. Additionally, we give a new object scale evaluation mechanism, which merely considers the differences between the maximum response values in consecutive frames. Extensive experiments on OTB2013 and OTB2015 datasets demonstrate that the proposed tracker performs favorably in relation to the state-of-the-art methods.

Improvement of the KCF Tracking Algorithm through Object Detection

When the position of the beam projector is changed, users have to manually adjust the position. In this paper, we propose a system that can automatically correct images. In this process, the KCF (Kernelized Correlation Filter) algorithm is used for tracking the IR (Infrared) markers. We analyze the object tracking failure problem of the KCF and improve the KCF tracking algorithm that solves the problem through object detection.

Joint Source and Channel Rate Allocation over Noisy Channels in a Vehicle Tracking Multimedia Internet of Things System.

As an emerging type of Internet of Things (IoT), multimedia IoT (MIoT) has been widely used in the domains of healthcare, smart buildings/homes, transportation and surveillance. In the mobile surveillance system for vehicle tracking, multiple mobile camera nodes capture and upload videos to a cloud server to track the target. Due to the random distribution and mobility of camera nodes, wireless networks are chosen for video transmission. However, the tracking precision can be decreased because of degradation of video quality caused by limited wireless transmission resources and transmission errors. In this paper, we propose a joint source and channel rate allocation scheme to optimize the performance of vehicle tracking in cloud servers. The proposed scheme considers the video content features that impact tracking precision for optimal rate allocation. To improve the reliability of data transmission and the real-time video communication, forward error correction is adopted in the application layer. Extensive experiments are conducted on videos from the Object Tracking Benchmark using the H.264/AVC standard and a kernelized correlation filter tracking scheme. The results show that the proposed scheme can allocate rates efficiently and provide high quality tracking service under the total transmission rate constraints.

Real-Time Obstacle Detection and Tracking for Sense-and-Avoid Mechanism in UAVs

Obstacle detection and tracking is an important research topic in computer vision with a number of practical applications. Though an ample amount of research has been done in this domain, implementing automatic obstacle detection and tracking in real time is still a big challenge. To address this issue, we propose a fast and robust obstacle detection and tracking approach by integrating an adaptive obstacle detection strategy within a kernelized correlation filter (KCF) framework in this paper. A suitable salient object detection method autoinitializes the KCF tracker for this purpose. Moreover, an adaptive obstacle detection strategy is proposed to refine the location and boundary of the object when the confidence value of the tracker drops below a predefined threshold. In addition, a reliable postprocessing technique is implemented to accurately localize the obstacle from a saliency map recovered from the search region. The proposed approach has been extensively tested through quantitative and qualitative evaluations on a number of challenging datasets. The experiments demonstrate that the proposed approach significantly outperforms the state-of-the-art methods in terms of tracking speed and accuracy.

Robust Tracking and Redetection: Collaboratively Modeling the Target and Its Context

Robust object tracking and redetection require stably predicting the trajectory of the target object and recovering from tracking failure by quickly redetecting it when it is lost during long-term tracking. The locations of the target and the background are calculated relative to the region occupied by the object. The effect of tracking can be enhanced by isolating the target and the background, modeling and tracking them, respectively, and integrating their tracking results. In this study, we propose an approach that builds motion models for the target and its context. Tracking results from a target tracker and a context tracker are integrated through linear fusion to predict the position of the target. A kernelized correlation filter tracker is used to track the target in the predicted position. When the target is lost, it can be quickly recovered by searching in the given field of view using a target model built and updated through observation models that are constructed prior to the loss of the target. Our approach is not sensitive to the segmentation of the target and the context. The motion models and observation models of the target and the context work together in the tracking process, whereas the target model alone is involved in redetection. Experiments to test our proposed approach, which simultaneously models the target and its context, showed that it can effectively enhance the robustness of long-term tracking.

Robust kernelized correlation filter with scale adaption for real-time single object tracking

Kernelized correlation filter (KCF) is a kind of efficient method for real-time tracking, but remains being challenged by the drifting problem due to inaccurate localization caused by the scale variation and wrong candidate selection. In this paper, we propose a new scale adaptive kernelized correlation filter tracker, termed as SKCF, which estimates an accurate scale and models the distribution of correlation response to address the template drifting problem. In SKCF, a scale adaption method is used to find an accurate candidate. Thus we improve its capacity to drastic scale change which usually happens for unmanned aerial vehicles (UAVs)-based applications. The SKCF also introduces a Gaussian distribution to model the correlation response of the target image to select a better candidate in tracking procedure. Extensive experiments are performed on two commonly used tracking benchmarks and also a new benchmark for UAV tracking with complex scale variations. The results show that the proposed SKCF significantly improves the performance compared to the baseline KCF and achieves better performance than state-of-the-art single object trackers at real-time.

Hybrid Head Tracking for Wheelchair Control Using Haar Cascade Classifier and KCF Tracker

Disability may limit someone to move freely, especially when the severity of the disability is high. In order to help disabled people control their wheelchair, head movement-based control is preferred due to its reliability. This paper proposed a head direction detector framework which can be applied to wheelchair control. First, face and nose were detected from a video frame using Haar cascade classfier. Then, the detected bounding boxes were used to initialize Kernelized Correlation Filters tracker. Direction of a head was determined by relative position of the nose to the face, extracted from tracker’s bounding boxes. Results show that the method effectively detect head direction indicated by 82% accuracy and very low detection or tracking failure.

Object Tracking in Satellite Videos by Fusing the Kernel Correlation Filter and the Three-Frame-Difference Algorithm

Object tracking is a popular topic in the field of computer vision. The detailed spatial information provided by a very high resolution remote sensing sensor makes it possible to track targets of interest in satellite videos. In recent years, correlation filters have yielded promising results. However, in terms of dealing with object tracking in satellite videos, the kernel correlation filter (KCF) tracker achieves poor results due to the fact that the size of each target is too small compared with the entire image, and the target and the background are very similar. Therefore, in this letter, we propose a new object tracking method for satellite videos by fusing the KCF tracker and a three-frame-difference algorithm. A specific strategy is proposed herein for taking advantage of the KCF tracker and the three-frame-difference algorithm to build a strong tracker. We evaluate the proposed method in three satellite videos and show its superiority to other state-of-the-art tracking methods.