Object Tracking Model Research Articles

Towards a New Paradigm for Brain-inspired Computer Vision

Brain-inspired computer vision aims to learn from biological systems to develop advanced image processing techniques. However, its progress so far is not impressing. We recognize that a main obstacle comes from that the current paradigm for brain-inspired computer vision has not captured the fundamental nature of biological vision, i.e., the biological vision is targeted for processing spatio-temporal patterns. Recently, a new paradigm for developing brain-inspired computer vision is emerging, which emphasizes on the spatio-temporal nature of visual signals and the brain-inspired models for processing this type of data. In this paper, we review some recent primary works towards this new paradigm, including the development of spike cameras which acquire spiking signals directly from visual scenes, and the development of computational models learned from neural systems that are specialized to process spatio-temporal patterns, including models for object detection, tracking, and recognition. We also discuss about the future directions to improve the paradigm.

Kalman Filtering and Bipartite Matching Based Super-Chained Tracker Model for Online Multi Object Tracking in Video Sequences

Object tracking has gained importance in various applications especially in traffic monitoring, surveillance and security, people tracking, etc. Previous methods of multiobject tracking (MOT) carry out detections and perform object tracking. Although not optimal, these frameworks perform the detection and association of objects with feature extraction separately. In this article, we have proposed a Super Chained Tracker (SCT) model, which is convenient and online and provides better results when compared with existing MOT methods. The proposed model comprises subtasks, object detection, feature manipulation, and using representation learning into one end-to-end solution. It takes adjacent frames as input, converting each frame into bounding boxes’ pairs and chaining them up with Intersection over Union (IoU), Kalman filtering, and bipartite matching. Attention is made by object attention, which is in paired box regression branch, caused by the module of object detection, and a module of ID verification creates identity attention. The detections from these branches are linked together by IoU matching, Kalman filtering, and bipartite matching. This makes our SCT speedy, simple, and effective enough to achieve a Multiobject Tracking Accuracy (MOTA) of 68.4% and Identity F1 (IDF1) of 64.3% on the MOT16 dataset. We have studied existing tracking techniques and analyzed their performance in this work. We have achieved more qualitative and quantitative tracking results than other existing techniques with relatively improved margins.

Multiple Object Tracking of Drone Videos by a Temporal-Association Network with Separated-Tasks Structure

The task of multi-object tracking via deep learning methods for UAV videos has become an important research direction. However, with some current multiple object tracking methods, the relationship between object detection and tracking is not well handled, and decisions on how to make good use of temporal information can affect tracking performance as well. To improve the performance of multi-object tracking, this paper proposes an improved multiple object tracking model based on FairMOT. The proposed model contains a structure to separate the detection and ReID heads to decrease the influence between every function head. Additionally, we develop a temporal embedding structure to strengthen the representational ability of the model. By combing the temporal-association structure and separating different function heads, the model’s performance in object detection and tracking tasks is improved, which has been verified on the VisDrone2019 dataset. Compared with the original method, the proposed model improves MOTA by 4.9% and MOTP by 1.2% and has better tracking performance than the models such as SORT and HDHNet on the UAV video dataset.

Hybridization of Deep Convolutional Neural Network for Underwater Object Detection and Tracking Model

In this present work, underwater object detection and tracking was studied using the efficient Hybridization of Deep Convolutional Neural Network for Underwater Object Detection and Tracking (HDCNN-UODT) model for three bench mark data sets namely UOT32, brackish, and URPC 2020 datasets. The HDCNN-UODT technique primarily employs data augmentation process for increasing the size of the training dataset to improve the detection and average precision. Besides, a hybridization of two Deep Learning (DL) models namely RetinaNet and EfficientNet models was applied as feature extractors. In addition, the bounding box prediction process takes place via support vector regression (SVR) followed by kernel extreme learning machine (KELM) model. The novelty of the present work was demonstrated by the design of SVR based bounding box regression and fusion based feature extractions. The average precision examination (APE), average success rate(ASR) & average frame per second(AFPS) analysis of the HDCNN-UODT model using UOT32 dataset showed better APE (51.27%), ASR (43.19) & AFPS (310.25) over other reported techniques. Additionally, the detection accuracy of the HDCNN-UODT model for brackish and URPC datasets showed improved accuracies for different objects over YOLO v4, T-YOLO v4, and AFFM-YOLO v4 techniques. Moreover, using brackish dataset, HDCNN-UODT model showed highest accuracy of 94.85% for ‘Crab’ object and for URPC dataset, a highest accuracy of 88.34% for ‘Scallop’ object was obtained. Hence based on our outcome, HDCNN-UODT technique might be better suited for the object detection and tracking application.

Experimental study on the control of a suspended cable-driven parallel robot for object tracking purpose

AbstractIn this paper, control of a suspended cable-driven parallel robot has been experimentally investigated based on the dynamic model of the robot for object tracking purpose. In order to improve the tracking ability of the robot, three control approaches, namely kinematic PID, dynamic PD, and a kinematic sliding mode control (SMC), have been implemented, both on the Simscape and on the robot constructed at the Human and Robot Interaction Laboratory. Neural network controller and dynamic SMC have been implemented on the Simscape model. Afterward, the effectiveness of each approach has been investigated by employing the root mean square error (RMSE) index. Simulation and experimental results reveal the ability of each controller for precise and smooth control. For precise and real-time object tracking, YOLOv5-s and YOLOv4-tiny model are trained. By comparing the obtained index values, the kinematic PID demonstrates the best performance with the maximum RMSE value of 0.018 compared to other methods.

Computational Intelligence-Based Harmony Search Algorithm for Real-Time Object Detection and Tracking in Video Surveillance Systems

Recently, video surveillance systems have gained significant interest in several application areas. The examination of video sequences for the detection and tracking of objects remains a major issue in the field of image processing and computer vision. The object detection and tracking process includes the extraction of moving objects from the frames and continual tracking over time. The latest advances in computation intelligence (CI) techniques have become popular in the field of image processing and computer vision. In this aspect, this study introduces a novel computational intelligence-based harmony search algorithm for real-time object detection and tracking (CIHSA-RTODT) technique on video surveillance systems. The CIHSA-RTODT technique mainly focuses on detecting and tracking the objects that exist in the video frame. The CIHSA-RTODT technique incorporates an improved RefineDet-based object detection module, which can effectually recognize multiple objects in the video frame. In addition, the hyperparameter values of the improved RefineDet model are adjusted by the use of the Adagrad optimizer. Moreover, a harmony search algorithm (HSA) with a twin support vector machine (TWSVM) model is employed for object classification. The design of optimal RefineDet feature extraction with the application of HSA to appropriately adjust the parameters involved in the TWSVM model for object detection and tracking shows the novelty of the work. A wide range of experimental analyses are carried out on an open access dataset, and the results are inspected in several ways. The simulation outcome reported the superiority of the CIHSA-RTODT technique over the other existing techniques.

Object Tracking in Satellite Videos: A Spatial-Temporal Regularized Correlation Filter Tracking Method With Interacting Multiple Model

Target occlusion is common in satellite videos, which makes object tracking difficult because most state-of-the-art trackers are not robust to occlusion, particularly complete occlusion. In this letter, we propose a novel correlation filter algorithm with an interacting multiple model (IMM) for object tracking in satellite videos that combines the strength of the correlation filter and the IMM. When the target is occluded, we utilize the IMM to predict target position. Therefore, the proposed tracker is robust to occlusion. The experimental results demonstrate that our tracker performs favorably when the target is occluded and achieves excellent performance compared with state-of-the-art methods.

HM-Net: A Regression Network for Object Center Detection and Tracking on Wide Area Motion Imagery

Wide Area Motion Imagery (WAMI) yields high resolution images with a large number of extremely small objects. Target objects have large spatial displacements throughout consecutive frames. This nature of WAMI images makes object tracking and detection challenging. In this paper, we present our deep neural network-based combined object detection and tracking model, namely, Heat Map Network (HM-Net). HM-Net is significantly faster than state-of-the-art frame differencing and background subtraction-based methods, without compromising detection and tracking performances. HM-Net follows object center-based joint detection and tracking paradigm. Simple heat map-based predictions support unlimited number of simultaneous detections. The proposed method uses two consecutive frames and the object detection heat map obtained from the previous frame as input, which helps HM-Net monitor spatio-temporal changes between frames and keeps track of previously predicted objects. Although reuse of prior object detection heat map acts as a vital feedback-based memory element, it can lead to unintended surge of false positive detections. To increase robustness of the method against false positives and to eliminate low confidence detections, HM-Net employs novel feedback filters and advanced data augmentations. HM-Net outperforms state-of-the-art WAMI moving object detection and tracking methods on WPAFB dataset with its 96.2% F1 and 94.4% mAP detection scores, while achieving a 61.8% mAP tracking score on the same dataset.

Deep Learning Enabled Object Detection and Tracking Model for Big Data Environment

Recently, big data becomes evitable due to massive increase in the generation of data in real time application. Presently, object detection and tracking applications becomes popular among research communities and finds useful in different applications namely vehicle navigation, augmented reality, surveillance, etc. This paper introduces an effective deep learning based object tracker using Automated Image Annotation with Inception v2 based Faster RCNN (AIA-IFRCNN) model in big data environment. The AIA-IFRCNN model annotates the images by Discriminative Correlation Filter (DCF) with Channel and Spatial Reliability tracker (CSR), named DCF-CSRT model. The AIA-IFRCNN technique employs Faster RCNN for object detection and tracking, which comprises region proposal network (RPN) and Fast R-CNN. In addition, inception v2 model is applied as a shared convolution neural network (CNN) to generate the feature map. Lastly, softmax layer is applied to perform classification task. The effectiveness of the AIA-IFRCNN method undergoes experimentation against a benchmark dataset and the results are assessed under diverse aspects with maximum detection accuracy of 97.77%.

Methods of Passing Image to Object Tracking Model

Methods of Passing Image to Object Tracking Model

Integration Between Cascade Region-Based Convolutional Neural Network and Bi-Directional Feature Pyramid Network for Live Object Tracking and Detection

The current target tracking and detection algorithms often have mistakes and omissions when the target is occluded or small. To overcome the defects, this paper integrates bi-directional feature pyramid network (BiFPN) into cascade region-based convolutional neural network (R-CNN) for live object tracking and detection. Specifically, the BiFPN structure was utilized to connect between scales and fuse weighted features more efficiently, thereby enhancing the network’s feature extraction ability, and improving the detection effect on occluded and small targets. The proposed method, i.e., Cascade R-CNN fused with BiFPN, was compared with target detection algorithms like Cascade R-CNN and single shot detection (SSD) on a video frame dataset of wild animals. Our method achieved a mean average precision (mAP) of 91%, higher than that of SSD and Cascade R-CNN. Besides, it only took 0.42s for our method to detect each image, i.e., the real-time detection was realized. Experimental results prove that the proposed live object tracking and detection model, i.e., Cascade R-CNN fused with BiFPN, can adapt well to the complex detection environment, and achieve an excellent detection effect.

Auto-attentional mechanism in multi-domain convolutional neural networks for improving object tracking

PurposeMulti-domain convolutional neural network (MDCNN) model has been widely used in object recognition and tracking in the field of computer vision. However, if the objects to be tracked move rapid or the appearances of moving objects vary dramatically, the conventional MDCNN model will suffer from the model drift problem. To solve such problem in tracking rapid objects under limiting environment for MDCNN model, this paper proposed an auto-attentional mechanism-based MDCNN (AA-MDCNN) model for the rapid moving and changing objects tracking under limiting environment.Design/methodology/approachFirst, to distinguish the foreground object between background and other similar objects, the auto-attentional mechanism is used to selectively aggregate the weighted summation of all feature maps to make the similar features related to each other. Then, the bidirectional gated recurrent unit (Bi-GRU) architecture is used to integrate all the feature maps to selectively emphasize the importance of the correlated feature maps. Finally, the final feature map is obtained by fusion the above two feature maps for object tracking. In addition, a composite loss function is constructed to solve the similar but different attribute sequences tracking using conventional MDCNN model.FindingsIn order to validate the effectiveness and feasibility of the proposed AA-MDCNN model, this paper used ImageNet-Vid dataset to train the object tracking model, and the OTB-50 dataset is used to validate the AA-MDCNN tracking model. Experimental results have shown that the augmentation of auto-attentional mechanism will improve the accuracy rate 2.75% and success rate 2.41%, respectively. In addition, the authors also selected six complex tracking scenarios in OTB-50 dataset; over eleven attributes have been validated that the proposed AA-MDCNN model outperformed than the comparative models over nine attributes. In addition, except for the scenario of multi-objects moving with each other, the proposed AA-MDCNN model solved the majority rapid moving objects tracking scenarios and outperformed than the comparative models on such complex scenarios.Originality/valueThis paper introduced the auto-attentional mechanism into MDCNN model and adopted Bi-GRU architecture to extract key features. By using the proposed AA-MDCNN model, rapid object tracking under complex background, motion blur and occlusion objects has better effect, and such model is expected to be further applied to the rapid object tracking in the real world.

Sequence-tracker: Multiple object tracking with sequence features in severe occlusion scene

Multiple object tracking is one of the most fundamental tasks in computer vision, and it is still very challenging for real-world applications due to its severe occlusion and motion blur. Most of the existing methods solve these multiple object tracking issues by performing data association based on the deep features of the detections in consecutive frames, which only contain the spatial information of the detected objects. Therefore, the inaccuracy of data association would easily occur, especially in the severe occlusion scenes. In this paper, a novel multiple object tracking model named sequence-tracker (STracker) has been proposed, which combines both the temporal and spatial features to perform data association. We trained a sequence feature extraction network based on video pedestrian re-identification offline, fused the obtained sequence features with the depth features of the previous frame, and then implemented the Hungarian algorithm for data association. Experiments have been carried out to validate the effectiveness of the proposed algorithm and the corresponding results indicates that it can significantly improve the trajectory quality of our dataset in this paper. Remarkably, for the public detector results from MOT official website, the proposed algorithm can achieve up to 57.2% MOTA and 50.9% IDF1 on the MOT17 dataset.

Adaptive fuzzy weighted color histogram and HOG appearance model for object tracking with a dynamic trained neural network prediction

Object tracking in video sequences represents an important task in computer vision applications like human mobility and social distance in pandemic scenarios, intelligent surveillance systems, etc. This paper presents a new method, fuzzy weighted color histogram neural network (FWCHNN) to estimate the position of an object and to update the appearance object model during tracking. The method involves two stages. Prediction is achieved through an online dynamic trained neural network, and correction is performed with a spiral sliding window. The method employs a new similitude appearance measure based on a codified model of the object considering its shape and color. The shape is represented with HOG features and the color features from information of a fuzzy weighted color histogram. FWCHNN shows the comparable results regarding state-of-the-art methods.

Overlapping neural representations for the position of visible and imagined objects

Humans can covertly track the position of an object, even if the object is temporarily occluded. What are the neural mechanisms underlying our capacity to track moving objects when there is no physical stimulus for the brain to track? One possibility is that the brain ‘fills-in’ information about imagined objects using internally generated representations similar to those generated by feed-forward perceptual mechanisms. Alternatively, the brain might deploy a higher order mechanism, for example using an object tracking model that integrates visual signals and motion dynamics. In the present study, we used EEG and time-resolved multivariate pattern analyses to investigate the spatial processing of visible and imagined objects. Participants tracked an object that moved in discrete steps around fixation, occupying six consecutive locations. They were asked to imagine that the object continued on the same trajectory after it disappeared and move their attention to the corresponding positions. Time-resolved decoding of EEG data revealed that the location of the visible stimuli could be decoded shortly after image onset, consistent with early retinotopic visual processes. For processing of unseen/imagined positions, the patterns of neural activity resembled stimulus-driven mid-level visual processes, but were detected earlier than perceptual mechanisms, implicating an anticipatory and more variable tracking mechanism. Encoding models revealed that spatial representations were much weaker for imagined than visible stimuli. Monitoring the position of imagined objects thus utilises similar perceptual and attentional processes as monitoring objects that are actually present, but with different temporal dynamics. These results indicate that internally generated representations rely on top-down processes, and their timing is influenced by the predictability of the stimulus.

Edge-based Video Analytic for Smart Cities

Video analytic is the important tool for smart city development. The video analytic application requires more memories and high processing devices. The problems of cloud-based approach for video analytic are high latency and more network bandwidth to transfer data into the cloud. To overcome these problems, we propose a model based on dividing the jobs into smaller sub-tasks with less processing requirements in a typical video analytics application for the development of smart city. The object detection, tracking and pattern recognition method to reduce the size of videos based on edge network will be proposed. We will design a video analytic model, and simulation is performed using iFogSim simulator. We will also propose Convolutional Neural Network (CNN) based object tracking model. The experimental verification shows that our tracking model is more than 96% accurate, and the proposed edge and cloud-based model is more than 80% effective than only cloud-based approach for video analytic applications.

Object tracking using motion flow projection for pan-tilt configuration

We propose a new object tracking model for two degrees of freedom mechanism. Our model uses a reverse projection from a camera plane to a world plane. Here, the model takes advantage of optic flow technique by re-projecting the flow vectors from the image space into world space. A pan-tilt (PT) mounting system is used to verify the performance of our model and maintain the tracked object within a region of interest (ROI). This system contains two servo motors to enable a webcam rotating along PT axes. The PT rotation angles are estimated based on a rigid transformation of the the optic flow vectors in which an idealized translation matrix followed by two rotational matrices around PT axes are used. Our model was tested and evaluated using different objects with different motions. The results reveal that our model can keep the target object within a certain region in the camera view.

Multiple Reliable Structured Patches for Object Tracking

It is essential to build the effective appearance model for object tracking in computer vision. Most object trackers can be roughly divided into two categories according to the appearance model: the bounding box model and the patch model. The bounding box model cannot handle shape deformation and occlusion of the non-rigid moving object effectively. The patch model is prone to be disturbed by complex backgrounds. In this paper, we propose a robust multi-structured-patch appearance model to represent the target for object tracking. The proposed appearance model is aimed to exploit and identify reliable patches that can be tracked effectively through the whole tracking process. According to attention mechanism in biological vision system, a coarse-to-fine strategy is usually used to search the target. Therefore, the proposed appearance model is represented by robust patches in different sizes, in which the bigger patches search the rough region of the target and the smaller patches estimate the accurate location. Experimental results on OTB100 dataset show that the proposed method outperforms state-of-the-art trackers.

Learning multiple instance deep quality representation for robust object tracking

Robustly tracking various objects within a video stream with complex objects and backgrounds is a useful technique in next generation computer vision systems. However, in practice, it is difficult to design a successful video-based object tracking system due to the varied light conditions, possible occlusions, and fast-moving objects. In this work, a novel weakly-supervised and quality-guided visual object tracking model is proposed, wherein the key is a bidirectional long short-term memory recurrent neural network (BLSTM-RNN) that captures the feature sequence and predicts the quality score of each candidate window. More specifically, given a rich set of training videos annotated with the target objects, a weakly-supervised learning algorithm is first used to project all the candidate window features onto the semantic space. Next, we propose a two-stage algorithm to select the key frames from the video sequences, where both the shallow and deep filtering operations are conducted. Subsequently, the so-called BLSTM-RNN is proposed to characterize the feature sequence temporally, based on which the maximally possible object window can be calculated and finally output. In our experiment, a large video dataset containing 2045 NBA regular seasons and playoff basketball games was compiled. Based on this, a comparative study is conducted between the proposed algorithm and state-of-the-art video tracking methods. Extensive visualization results and comparative tracking precisions show the competitiveness of the proposed method.

A novel real-time object tracking based on kernelized correlation filter with self-adaptive scale computation in combination with color attribution

Recently, object tracking model based on correlation filter has achieved great success. However, most of these methods relying on correlation filter train many classifiers individually, and fail to cope with significant appearance and scale change of object and complex challenging environment. How to accurately and robustly compute object scale relative to template in visual object tracking application is a challenge topic at home and abroad. Many existing tracking algorithms are not very good at big scale variation in complex video. In order to solve the problem, a novel and effective scale computation algorithm based on tracking in detection is proposed in paper. The tracking object is divided into four parts, and compute their scale factors, then get the maximum response location by adopting kernelized correlation filter based on color attribution. Specially, aimed at the difficulty of object location, weighted coefficients are used to remove the abnormal matching points and the desired training output from common classifier is transformed into other forms. Simulation experiment result shows our proposed algorithm outperform others algorithms in color video with big scale variation.