Tracking-Learning-Detection Research Articles

Design of a Vision Based Autonomous Turret

This article describes the hardware and software design of a vision-based autonomous turret system. A two degree of freedom (2 DOF) turret platform is designed to carry a canon equipped with an embedded camera and actuated by stepper motors or direct current motors. The turret system includes a central calculator running a visual detection and tracking solution, and a microcontroller, responsible for actuators control. The Tracking-Learning-Detection (TLD) algorithm is implemented for target detection and tracking. Furthermore, a Kalman filter algorithm is implemented to continue the tracking in case of occlusion. The performances of the designed turret, regarding response time, accuracy and the execution time of its main tasks, are evaluated. In addition, an experimental scenario was performed for real-time autonomous detection and tracking of a moving target.

Magnetic Untethered Peanutlike Millirobot Motion Mode Switching Control

A brand new magnetic untethered peanutlike millirobot (MUPM) was created with two different motion modes—rolling motion and crablike motion, and designed to be actuated under a rotating magnetic field generated by a magnetic driving navigation system (MDNS). The dynamic characteristics of the MUPM in the different motion modes were studied through modeling. By adjusting the frequency of the rotating magnetic field, the movement velocity of the MUPM could be controlled and motion mode switching initiated, while the direction of the motion was controlled by regulating the rotating magnetic field direction. The mechanism of motion mode transitions was then analyzed and simulated. The MDNS allows the MUPM to navigate along a preset trajectory from a starting point to a destination, and tracking-learning-detection (TLD) algorithm achieved accurate tracking of the MUPM. It was demonstrated that the MUPM has excellent maneuverability in both its motion modes and can pass through a narrow channel to a targeted area by switching modes. The obstacle-overcoming ability of MUPM in two motion modes is tested. The MUPM is expected to have great application value in the field of targeted intraluminal drug delivery.

Augmented reality for visualization the narrow areas in jaw surgery: modified Correntropy based enhanced ICP algorithm

Over time, Augmented Reality (AR) based technology becomes not being properly to implement with oral and maxillofacial surgery to visualise the narrow area spot in jaw surgery as blood vassals and root canals in these types of surgeries. Image registration is considered the major limitation of using the AR in these types of surgeries and reduces the accuracy of visualising the narrow areas. In this research, we propose a Correntropy based scale ICP algorithm as a solution to improve the image registration during jaw surgery. Correntropy is considered here to minimise the error metric of the ICP algorithm instead of the Euclidean distance measurement compared to the state-of-the-art solution. This led to decrease the registration error, increase the video accuracy and reduce the processing time simultaneously. The proposed system consists of Enhanced Tracking Learning Detection (TLD), which is used as an occlusion removal featured algorithm in the intra-operative stage of the AR-based jaw surgery system. In this research, a Modified Correntropy-based enhanced ICP (MCbeICP) algorithm is proposed for the system’s pose-refinement phase. Moreover, this proposed algorithm (MCbeICP) has a new function to process the point set registration with great noises and outliers. It eliminates the poor performance of the ICP algorithm of the noisy point set. Furthermore, the ICP algorithm considers the scale factor to register the point with different scales of the real-time video and the sample models. Additionally, this method improves the result of the pose refinement stage in terms of registration accuracy and processing time. By this method, the pose refinement stage gives an improved result in terms of registration accuracy and processing time. The samples, which were taken from the upper (maxillary) and the lower (mandible) jaw bone show that the proposed algorithm provides a significant accuracy improvement in alignment to 0.21- 0.29 mm from 0.23 to 0.35 mm and an increment in processing time from 8 to 12 frames per second (fs/s) to 10-14 fs/s compared to the result provided by state of the art. The proposed augmented reality (AR) system is focused on the overlay accuracy and processing time. Finally, this study addressed the limitation of Image registration with AR using modified Correntropy-based enhanced ICP algorithm to implement oral and maxillofacial surgery successfully.

Application Research on Roller Skater Detection, Tracking, and Trajectory Prediction Based on Video Stream

With the continuous progress of science and technology, the sport of roller skating has developed rapidly and the technical level of the game has become higher and higher. Its sport performance has been rapidly improved. However, China’s roller skating is relatively late, and there is still a certain gap compared with many Western developed countries. In order to improve the performance of China’s roller skating, this study takes the representative Chinese and foreign excellent speed skaters as the research object and compares the sprinting technology of Chinese and foreign excellent speed skaters by using image measurement and image analysis to obtain the kinematic parameters and data of the athletes’ sprinting technology in the competition state. In view of the problem that the current video target tracking algorithm is easy to follow multiple targets, a video multiobject detection and tracking algorithm with improved tracking learning detection (TLD) is studied with the skater in the video as the research object. For the lost target, the prediction function of Kalman filter algorithm is used to track the trajectory of the typical target in the video, and the trajectory tracked by Kalman filter algorithm is used to compensate the lost part of TLD algorithm, so as to obtain the complete trajectory of the typical target in the video to improve the accuracy of video multiobject tracking. Since the existing trajectory prediction algorithms have the limitation of poor accuracy, a social-long short-term memory (Social-LSTM) network-based video typical target trajectory prediction algorithm is proposed to predict the trajectory sequences of typical targets to be detected by incorporating the contextual environment information and the interaction relationship between multiple target trajectories into the Social-LSTM network. The simulation results show that the proposed trajectory prediction algorithm outperforms the traditional LSTM algorithm, Hidden Markov Model Algorithm, and Hybrid Gaussian model algorithm, which is helpful to improve the accuracy of video roller skater target trajectory prediction, and the tracking success rate is 0.98.

Object Tracking in Video Using the TLD and CMT Fusion Model

Object tracking has been an attractive study topic in computer vision in recent years, thanks to the development of video monitoring systems. Tracking-Learning Detection (TLD), Compressive Tracking (CT), and Clustering of Static-Adaptive Correspondences for Deformable Object Tracking are some of the state-of-the-art methods for motion object tracking (CMT). We present a fusion model that combines TLD and CMT in this study. To restrict the calculation time of the CMT technique, the fusion TLD CMT model enhanced the TLD benefits of computation time and accuracy on t no deformable objects. The experimental results on the Vojir dataset for three techniques (TLD, CMT, and TLD CMT) demonstrated that our fusion proposal successfully trades off CMT accuracy for computing time.

Enhanced TLD-based video object-tracking implementation tested on embedded platforms

Object-tracking algorithms on embedded platforms are very important in many civilian and military applications. The Tracking–Learning–Detection (TLD) algorithm is considered one of the state-of-the-art online long-term object-tracking algorithms. The performance of running such computationally intensive algorithms on embedded platforms with limited computing resources is a challenge. This work proposes an enhanced TLD implementation, specifically designed for, and tested on, embedded platforms. In this new implementation, an extra-stage has been added to the TLD detector cascade, called a Region filter. This filter dynamically identifies the candidate region for the tracked object. Further, the two independent tracker and detector TLD components, and the two independent Forward–Backward (FB) and Normalized Cross Correlation (NCC) error measures in the tracker have been parallelized. Still further, the computations of Image Integral in the detector and the NCC in both the tracker and the detector have been optimized using a single instruction multiple data (SIMD) architecture. We evaluate our proposed implementation on the Apalis T30 embedded platform, using the same video sequences that the original TLD is evaluated on. Our results show that our enhanced implementation outperforms the baseline with an average speedup of 3.7 × in the total number of frames per second (fps), while achieving an average 91% of the Precision and 86.6% of the Recall metrics, across all sequences. Further, our enhanced implementation achieves an average speedup of 4.52 × and 1.86 × in the detector and tracker execution times, respectively. Moreover, it results in an average 66.3% energy saving, as compared to the original implementation.

Visual servoing of robot manipulator with weak field-of-view constraints

Aiming at the problem of servoing task failure caused by the manipulated object deviating from the camera field-of-view (FOV) during the robot manipulator visual servoing (VS) process, a new VS method based on an improved tracking learning detection (TLD) algorithm is proposed in this article, which allows the manipulated object to deviate from the camera FOV in several continuous frames and maintains the smoothness of the robot manipulator motion during VS. Firstly, to implement the robot manipulator visual object tracking task with strong robustness under the weak FOV constraints, an improved TLD algorithm is proposed. Then, the algorithm is used to extract the image features (object in the camera FOV) or predict image features (object out of the camera FOV) of the manipulated object in the current frame. And then, the position of the manipulated object in the current image is further estimated. Finally, the visual sliding mode control law is designed according to the image feature errors to control the motion of the robot manipulator so as to complete the visual tracking task of the robot manipulator to the manipulated object in complex natural scenes with high robustness. Several robot manipulator VS experiments were conducted on a six-degrees-of-freedom MOTOMANSV3 industrial manipulator under different natural scenes. The experimental results show that the proposed robot manipulator VS method can relax the FOV constraint requirements on real-time visibility of manipulated object and effectively solve the problem of servoing task failure caused by the object deviating from the camera FOV during the VS.

Multi-target video-based face recognition and gesture recognition based on enhanced detection and multi-trajectory incremental learning.

BACKGROUND: Video-based face recognition (VFR) is one of the frontier topics in the domain of computer vision, which aims to automatically track and recognize facial regions of interests (ROIs) in video sequences.OBJECTIVE: In videos with multiple faces, the trajectories of individuals are incredibly complex. This is less studied than videos with a single face per frame.METHODS: In this paper, we present a multi-trajectory incremental learning (MTIL) algorithm, which categorizes trajectories using a Euclidean distance-based greedy algorithm and estimates the most likely labels for each trajectory by incremental learning to correct their classification and improve the accuracy of recognition. Furthermore, this study proposes an enhanced detection method that combines face detection with a robust tracking-learning-detection (TLD) algorithm to improve the performance of face detection in video. The method can also be extended for medical video recognition applications such as gesture recognition control based medical system.RESULTS: Experiments on Honda/UCSD and BMP (seq_mb) database demonstrate that our method can improve the face detection and face recognition (single or multiple) performance. The method also performs well on the gesture recognition system.CONCLUSION: The proposed MTIL algorithm can significantly improve the performance of the VFR system and the gesture recognition system.

Stable Infrared Target Tracking Algorithm Under Complicated Background

During the long-term tracking process of a single infrared target,many technical problems occur,such as strong background interference,occlusion,deformation,and target feature attenuation.An infrared target-tracking algorithm based on tracking-learning-detection(TLD)was proposed to solve these problems.Based on compressive tracking(CT),generalized Harr-like features were replaced by histograms of oriented gradient features.In our proposed method,a complementary random measurement matrix,which extracted texture and optimized grayscale feature-weights,was introduced.Moreover,a Kalman filter,used to record the space context location information,was adopted.Hence,the tracking failure and global retrieval problem of traditional CT and TLD algorithms can be solved when the target is occluded or deformed.The infrared image-tracking algorithm based on the combination of the TLD algorithm framework and improved CT algorithm effectively solves the problem of occlusion and strong interference and improves the tracking accuracy and long-term tracking stability of the algorithm.Experimental results show that the proposed algorithm can track well in real time and maintain good accuracy and robustness in an infrared ground environment.

A Visual Object Tracking Algorithm Based on Improved TLD

Visual object tracking is an important research topic in the field of computer vision. Tracking–learning–detection (TLD) decomposes the tracking problem into three modules—tracking, learning, and detection—which provides effective ideas for solving the tracking problem. In order to improve the tracking performance of the TLD tracker, three improvements are proposed in this paper. The built-in tracking module is replaced with a kernelized correlation filter (KCF) algorithm based on the histogram of oriented gradient (HOG) descriptor in the tracking module. Failure detection is added for the response of KCF to identify whether KCF loses the target. A more specific detection area of the detection module is obtained through the estimated location provided by the tracking module. With the above operations, the scanning area of object detection is reduced, and a full frame search is required in the detection module if objects fails to be tracked in the tracking module. Comparative experiments were conducted on the object tracking benchmark (OTB) and the results showed that the tracking speed and accuracy was improved. Further, the TLD tracker performed better in different challenging scenarios with the proposed method, such as motion blur, occlusion, and environmental changes. Moreover, the improved TLD achieved outstanding tracking performance compared with common tracking algorithms.

Moving vehicle tracking based on improved tracking–learning–detection algorithm

This study addresses the tracking–learning–detection (TLD) algorithm for long‐term single‐target tracking of moving vehicle from video streams. The problems leading to tracking failures in existing TLD methods are discovered, and an improved TLD (ITLD) tracking algorithm is proposed which is more robust to object occlusion and illumination variation. A square root cubature Kalman filter (SRCKF) is employed in the tracker of TLD to predict the position of the object when occlusion occurs. Besides, this study introduces fast retina keypoint (FREAK) feature into the tracker to alleviate the instability caused by illumination variation or scale variation. The overlap comparison and the normalised cross‐correlation coefficient (NCC) are introduced to the integrator of the TLD to obtain reliable bounding boxes with improved tracking precision. Experiments are conducted to compare the performance of the state‐of‐the‐art trackers and the proposed method, using the object tracking benchmark that includes 50 video sequences (OTB‐50) and TLD datasets. The experimental results show that the proposed ITLD outperforms on both tracking accuracy and robustness. The proposed method can track a moving vehicle even when it is temporally totally occluded.

A new TLD target tracking method based on improved correlation filter and adaptive scale

Target tracking is a popular but challenging problem in computer vision field. Due to many disturbing factors such as position transformation, illumination, and occlusion, it is difficult to achieve continuous target tracking. On the basis of the above analyses, a novel target tracking method based on correlation filters is proposed in this paper. This method uses the improved Tracking–Learning–Detection (TLD) tracking framework which combines the tracker with the detector through the learning mechanism. In the TLD tracking framework, the Spatially Regularized Discriminatively Correlation Filters tracker is used and improved. In addition, the adaptive tracking scale is realized according to the confidence of the searching area. The experimental results show that the proposed algorithm can effectively deal with the attitude change and the illumination problem so that it has better robustness and stability for target continuous tracking.

Multi-object tracking in MRI-guided radiotherapy using the tracking-learning-detection framework

The application of the tracking-learning-detection (TLD) framework; a performant tracking algorithm for real-life objects in CCD video, was evaluated and successfully optimized for tracking anatomical structures in low-quality 2D cine-MRI acquired during MRI-guided radiotherapy. Sub-pixel tracking accuracy and >95% precision and recall was achieved despite significant deformations and periodical disappearance.

Fish Tracking Based on Improved TLD Algorithm in Real-World Underwater Environment

AbstractFish tracking in the natural underwater environment is of great value for many applications, such as fish behavior analysis and the estimation of fish population density. Due to the variety of swimming postures, uneven illumination, and complicated background in the real-world underwater environment, most existing object tracking algorithms are not ideal for fish tracking. In this paper, a tracking algorithm based on TLD (Tracking-Learning-Detection) is proposed, in which the shape and color features of fish are fully utilized to achieve accurate and rapid tracking. The proposed algorithm mainly focuses on the tracking module and detection module in the TLD framework. In the tracking module, a scale adaptive mean-shift algorithm is employed to deal with the variety of swimming postures and the uneven illumination by estimating the scale and using the color histogram of the background and target area. In the detection module, the Hu invariant moment of fish, which encodes the shape information, is used to detect the target quickly. The experiment results show that the proposed algorithm is effective and accurate for fish tracking in the real-world underwater environment.

DESIGN AND IMPLEMENTATION OF THE VISUAL DETECTION SYSTEM FOR AMPHIBIOUS ROBOTS

Aiming at application requirements of our spherical amphibiousrobot, which has one type of propulsion system to be used bothon land and in water, an improved tracking learning detection(TLD) algorithm is designed and implemented, so as to improve therobustness and environmental adaptability of its visual detectionsystem.

Human motion tracking: A comparative study

In this paper, we aim to carry out a brief study of a new tracking system and the results of existing tracking systems including Compressive Tracking (CT), Fragment Tracker (FRAG), Multiple Instance Learning Tracker (MIL), and Tracking Learning Detection algorithm (TLD). For this purpose, we present the principles of those tracking systems. Then, we test them on several datasets that contain challenging attributions. Finally, we evaluate them qualitatively and quantitatively by two widely used criterions: the Average Center Error (ACE) and the Average Overlap Error (AOE). We conclude our paper by discussing the obtained results and the future works in this field.

Real-Time Pedestrian Tracking and Counting with TLD

This paper describes a solution to solve the issue of automatic multipedestrian tracking and counting. First, background modeling algorithm is applied to actively obtain multipedestrian candidates, followed by a confirmation step with classification. Then each pedestrian patch is handled by real-time TLD (Tracking-Learning-Detection) to get a new predication position according to similarity measure. Further TLD results are compared with classification list to determine a new, disappeared, or existing pedestrian. Finally single line counting with buffer zone is employed to count pedestrians. Experiments results on the public database, PETS, demonstrate the validity of our solution.

Dynamic target tracking based on corner enhancement with Markov decision process

The tracking–learning–detection (TLD) algorithm applied in the home environment can effectively improve the tracking robustness. However, it has the problems of single target tracking and poor selection of feature points. This study proposed a dynamic target tracking method based on corner enhancement with Markov decision process (MDP) model. The MDP target tracking method is adopted to change a multi-target tracking problem into a strategy problem based on MDP model, in which one MDP model represents the life cycle of a target, and multiple targets are represented by multiple MDP models. In the tracking process, the strong corners generated by the Shi-Tomasi corner method are used to replace the feature points generated by the traditional TLD algorithm at intermediate intervals, which makes the target feature points more stable during the tracking process. The similarity function learning for data association is equivalent to the learning of the MDP strategy, in which the reinforcement learning method is used and has double advantages of both online and offline learning. The tracking experiments with different data sets are performed, and the results show that dynamic target tracking algorithm based on the corner enhancement with MDP has both good tracking performance and good anti-interference capability.

A long time tracking with BIN-NST and DRN

Long time tracking in video stream is one of the most challenging problem in computer vision. Most state-of-the-art visual trackers rely on the classical sliding windows, resulting in a large number of candidate object window. In order to achieve purpose of real-time tracking, the weak feature is used for object description. So in many complex scenes, such as illumination variation, the algorithm performance is seriously decreased. Firstly, a more efficient object detection algorithm called BIN-NST is used to improve the detector in the tracking–learning–detection framework. We find that the normed gradients, designed for generic objectness estimation, are also able to rapidly generate high quality candidate windows. We also notice that using the tensor feature to replace the gradients feature can improve the performance of objectness estimation. Based on these observations, we propose an efficient method, which we call BIN-NST, to produce candidate windows. Experiments on the challenging PASCAL VOC 2007 dataset show that our method efficiently generates a small set of category-independent, high quality object windows. The performance of BIN-NST and BING is almost the same in the single scene, but the performance of BIN-NST in the complex scene is significantly higher than that of the BING. In addition, compared to the classical sliding windows mechanism used in the tracking–learning–detection framework, the candidate window generated by BIN-NST is reduced by about 90%. Secondly, GOTURN tracker is a method for offline training of neural networks that can track novel objects at test-time at 100 fps. The GOTURN tracker is significantly faster than previous methods that use neural networks for tracking. But the GOTURN tracker will fail to track object when the object become occluded and is moving too quickly. In this paper, these candidate object windows, which are detected by object detector based on BIN-NST, are fed to the network of GOTURN. The object extraction ability of the detector is used to solve the problem that the GOTURN algorithm cannot track fast moving object effectively. The experiment shows that the tracking algorithm can still track the object effectively when the object is moving quickly.

Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks

Abstract Activity identification is an essential step to measure and monitor the performance of earthmoving operations. Many vision-based methods that automatically capture and explain activity information from image data have been developed with economic advantages and analysis efficiency. However, the previous methods failed to consider the interactive operations among equipment, and thus limited the applicability to the operation time estimation for productivity analysis. To address the drawback, this research developed a vision-based activity identification framework that incorporates interactive aspects of earthmoving equipment's operation. This framework included four main processes: equipment tracking, action recognition of individual equipment, interaction analysis, and post-processing. The interactions between excavators and dump trucks were examined due to its significant impacts on earthmoving operations. TLD (Tracking-Learning-Detection) was adapted to track the heavy equipment. Spatio-temporal reasoning and image differencing techniques were then implemented to categorize individual actions. Third, interactions were interpreted based on a knowledge-based system that evaluates equipment actions and proximity between operating equipment. Lastly, outliers or noisy results were filtered out considering work continuity. To validate the proposed framework, two experiments were performed: one with the interaction analysis and the other without the analysis. 11,513 image frames from actual earthmoving sites in total were tested. The consequent average precision of activity analysis was enhanced from 75.68% to 91.27% after the interaction analysis was applied. In conclusion, this research contributes to identifying critical elements that explain interactive operations, characterize the vision-based activity identification framework, and improve the applicability of the vision-based method for the automated equipment operations analysis.