Video Object Tracking Research Articles

Deep Siamese Network With Motion Fitting for Object Tracking in Satellite Videos

With the advancement in remote sensing satellite technology, object tracking in satellite videos has become an emerging research field. However, due to small object size, little appearance features, and poor distinguishability between targets and the background, traditional trackers with handcraft visual features achieve poor results in satellite videos. Deep neural networks have shown powerful potential for object tracking in ordinary videos but remain developing in satellite videos. In this letter, a Siamese network and a motion regression network are adopted to form a two-stream deep neural network (SRN) for satellite object tracking, which simultaneously utilizes appearance and motion features. Besides, a trajectory fitting motion (TFM) model based on history trajectories is also employed to further alleviate model drift. Comprehensive experiments demonstrate that the proposed method performs favorably compared with the state-of-the-art tracking methods.

Object Tracking in Satellite Videos: Correlation Particle Filter Tracking Method With Motion Estimation by Kalman Filter

Object tracking in satellite videos faces various challenges such as target occlusion, target rotation, and background clutter. This study proposes a correlation particle filter algorithm with motion estimation for object tracking in satellite videos. The tracker, called CPKF, combines the strengths of the correlation, particle, and Kalman filters. Compared with existing tracking methods based on correlation filters, the proposed tracker has three major advantages: (1) Particle sampling, and motion estimation build robustness against partial and complete occlusion. (2) Color histogram model makes it robust to target rotation. (3) Fusion of multiple feature response maps effectively handle background clutter and low contrast. The experimental results demonstrate that the proposed tracking algorithm performs better than state-of-the-art methods.

Multivehicle Object Tracking in Satellite Video Enhanced by Slow Features and Motion Features

With the development of video satellites, multimoving object tracking in satellite video is possible and has become a new challenging task. The difficulties are mainly caused by the characteristics of satellite videos: 1) small objects; 2) low contrast between objects and background; and 3) background in a state of continuous motion. These characteristics make it difficult for the advanced multiobject tracking algorithms in the natural video to give full play to their advantages, resulting in vast false alarms, missed objects, ID switches, and low-confidence bounding boxes. To tackle these problems, a novel multimoving object tracking method considering slow features (SFs) and motion features has been proposed in this research, named SF and motion feature-guided multiobject tracking (SFMFMOT), which realizes the continuous tracking of moving vehicles in satellite videos. A nonmaximum suppression (NMS) module guided by bounding box proposals based on SFs is designed to assist the object detection part by utilizing the sensitivity of SF analysis to the changed pixels. While removing a large number of static false alarms and supplementing missed objects, it improves the recall rate by increasing the confidence score of the correctly detected object bounding boxes. In order to improve the tracking performance, a set of optimization strategies based on motion features and time accumulation information are proposed to smooth the trajectory, remove static false alarms, and duplicate bounding boxes. The proposed method is evaluated in three satellite videos and its superiority is demonstrated.

Spatial–Spectral Weighted and Regularized Tensor Sparse Correlation Filter for Object Tracking in Hyperspectral Videos

Hyperspectral video camera captures spatial, spectral and temporal information of moving objects. Traditional object tracking methods developed for color videos have been applied to hyperspectral videos after compressing hundreds of spectral bands into three, which does not fully utilize the wealth spectral information. In order to address this issue, we present a tensor sparse correlation filter with a spatial-spectral weighted regularizer for object tracking. First, tensor processing is employed to reduce the spectral differences of homogeneous background, thereby producing robust spectral structure features. Second, a spatial-spectral weighted regularizer is designed in the correlation filter framework to penalize filter template by suppressing spectral features dissimilar to the center pixel in tracking. Third, a sparse constraint term and tracking context information are incorporated to suppress unexpected peaks in the response map. Finally, a reformulated stacked HOG feature extractor and a two-dimensional adaptive scale search strategy are developed to further improve the tracker’s feature discrimination and scale adaptation capability. Experimental results demonstrate that the proposed method achieves superior tracking performance than traditional correlation filter-based trackers.

Object Tracking in Satellite Videos: A Spatial-Temporal Regularized Correlation Filter Tracking Method With Interacting Multiple Model

Target occlusion is common in satellite videos, which makes object tracking difficult because most state-of-the-art trackers are not robust to occlusion, particularly complete occlusion. In this letter, we propose a novel correlation filter algorithm with an interacting multiple model (IMM) for object tracking in satellite videos that combines the strength of the correlation filter and the IMM. When the target is occluded, we utilize the IMM to predict target position. Therefore, the proposed tracker is robust to occlusion. The experimental results demonstrate that our tracker performs favorably when the target is occluded and achieves excellent performance compared with state-of-the-art methods.

FAANet: feature-aligned attention network for real-time multiple object tracking in UAV videos

FAANet: feature-aligned attention network for real-time multiple object tracking in UAV videos

Criss-Cross Attentional Siamese Networks for Object Tracking

Visual object tracking is a hot topic in recent years. In the meanwhile, Siamese networks have attracted extensive attention in this field because of its balanced precision and speed. However, most of the Siamese network methods can only distinguish foreground from the non-semantic background. The fine-tuning and retraining of fully-convolutional Siamese networks for object tracking(SiamFC) can achieve higher precision under interferences, but the tracking accuracy is still not ideal, especially in the environment with more target interferences, dim light, and shadows. In this paper, we propose criss-cross attentional Siamese networks for object tracking (SiamCC). To solve the imbalance between foreground and non-semantic background, we use the feature enhancement module of criss-cross attention to greatly improve the accuracy of video object tracking in dim light and shadow environments. Experimental results show that the maximum running speed of SiamCC in the object tracking benchmark dataset is 90 frames/second. In terms of detection accuracy, the accuracy of shadow sequences is greatly improved, especially the accuracy score of sequence HUMAN8 is improved from 0.09 to 0.89 compared with the original SiamFC, and the success rate score is improved from 0.07 to 0.55.

SatSOT: A Benchmark Dataset for Satellite Video Single Object Tracking

By imaging a specific area continuously, satellite video shows excellent capability in various applications such as surveillance and traffic management. Although object tracking has made significant progress in recent years, development in satellite object tracking is limited by the lack of open-source satellite datasets. It is thus essential to establish a satellite video object-tracking benchmark to fill the gap and advance the research. In this work, we present SatSOT, the first densely annotated satellite video single object-tracking benchmark dataset. SatSOT consists of 105 sequences with 27664 frames, 11 attributes, and four categories of typical moving targets in satellite videos: car, plane, ship, and train. Based on the proposed dataset and the significant challenges in satellite video object tracking, such as small targets, background interference, and severe occlusion, detailed evaluation and analysis are performed on 15 among the best and most representative tracking algorithms, which provides a basis for further research on satellite video object tracking.

АЛГОРИТМЫ МОНИТОРИНГА ПОЛОЖЕНИЯ ОБЪЕКТОВ НА МОРСКОЙ ПОВЕРХНОСТИ С ПОМОЩЬЮ МОНОКУЛЯРНОЙ ВИДЕОКАМЕРЫ

The article proposes new algorithms for estimating the coordinates of objects (both linear and angular) relative to the coordinate system related to the video camera. A two-step algorithm is proposed. At the first stage, the processing of images coming from the camera is performed – the selection of an area be-longing to the sea surface in the image, the detection and video tracking of objects, the determination of azimuth and elevation angles from the obtained images. Our approach is based on the representation of elevation and azimuth angles in the form of non-stationary autoregression models, recurrent estimation of their parameters and subsequent estimation of the object coordinates.

UAS Current Mapping: A Wave-Based Heading and Position Correction

AbstractOur unmanned aerial system (UAS) current mapping is based on optical video data of the sea surface. We use three-dimensional fast Fourier transform and least squares fitting to measure the surface waves’ phase velocities and the currents via the linear dispersion relationship. Our UAS is a low-cost off-the-shelf quadcopter with inaccurate camera position and attitude measurements, which may cause spurious currents as large as the signal. We present a novel wave-based UAS heading and position correction, improving the image rectification accuracy by a factor of ~3.5 and the current measurements’ temporal repeatability by factors of 1.8–4.8. This validation study maps the currents at high spatiotemporal resolution (5 m and 4 s) across the ~700-m-wide tidally dominated Bear Cut channel in Miami, Florida. The UAS currents are compared to flotsam tracks, obtained through automated UAS video object detection and tracking, drifter tracks, and acoustic Doppler current profiler measurements. The root-mean-square errors of the cross- and along-channel currents are better than 0.03 m s−1 for the flotsam comparison and better than 0.06 m s−1 for the drifter comparison; the latter revealed a 0.06 m s−1 along-wind bias due to wind- and wave-driven vertical current shear. UAS current mapping could be used to monitor river discharge, buoyant pollutants, or submesoscale fronts and eddies; the proposed wave-based heading and position correction enables its use in areas without ground control points.

A fast region-based active contour for non-rigid object tracking and its shape retrieval.

Conventional tracking approaches track objects using a rectangle bounding box. Gait, gesture and many medical analyses require non-rigid shape extraction. A non-rigid object tracking is more difficult because it needs more accurate object shape and background separation in contrast to rigid bounding boxes. Active contour plays a vital role in the retrieval of image shape. However, the large computation time involved in contour tracing makes its use challenging in video processing. This paper proposes a new formation of the region-based active contour model (ACM) using a mean-shift tracker for video object tracking and its shape retrieval. The removal of re-initialization and fast deformation of the contour is proposed to retrieve the shape of the desired object. A contour model is further modified using a mean-shift tracker to track and retrieve shape simultaneously. The experimental results and their comparative analysis concludes that the proposed contour-based tracking succeed to track and retrieve the shape of the object with 71.86% accuracy. The contour-based mean-shift tracker resolves the scale-orientation selection problem in non-rigid object tracking, and resolves the weakness of the erroneous localization of the object in the frame by the tracker.

Contrastive Transformation for Self-supervised Correspondence Learning

In this paper, we focus on the self-supervised learning of visual correspondence using unlabeled videos in the wild. Our method simultaneously considers intra- and inter-video representation associations for reliable correspondence estimation. The intra-video learning transforms the image contents across frames within a single video via the frame pair-wise affinity. To obtain the discriminative representation for instance-level separation, we go beyond the intra-video analysis and construct the inter-video affinity to facilitate the contrastive transformation across different videos. By forcing the transformation consistency between intra- and inter-video levels, the fine-grained correspondence associations are well preserved and the instance-level feature discrimination is effectively reinforced. Our simple framework outperforms the recent self-supervised correspondence methods on a range of visual tasks including video object tracking (VOT), video object segmentation (VOS), pose keypoint tracking, etc. It is worth mentioning that our method also surpasses the fully-supervised affinity representation (e.g., ResNet) and performs competitively against the recent fully-supervised algorithms designed for the specific tasks (e.g., VOT and VOS).

A Unified Multi-Scenario Attacking Network for Visual Object Tracking

Existing methods of adversarial attacks successfully generate adversarial examples to confuse Deep Neural Networks (DNNs) of image classification and object detection, resulting in wrong predictions. However, these methods are difficult to attack models of video object tracking, because the tracking algorithms could handle sequential information across video frames and the categories of targets tracked are normally unknown in advance. In this paper, we propose a Unified and Effective Network, named UEN, to attack visual object tracking models. There are several appealing characteristics of UEN: (1) UEN could produce various invisible adversarial perturbations according to different attack settings by using only one simple end-to-end network with three ingenious loss function; (2) UEN could generate general visible adversarial patch patterns to attack the advanced trackers in the real-world; (3) Extensive experiments show that UEN is able to attack many state-of-the-art trackers effectively (e.g. SiamRPN-based networks and DiMP) on popular tracking datasets including OTB100, UAV123, and GOT10K, making online real-time attacks possible. The attack results outperform the introduced baseline in terms of attacking ability and attacking efficiency.

Object Tracking in Hyperspectral-Oriented Video with Fast Spatial-Spectral Features

This paper presents a correlation filter object tracker based on fast spatial-spectral features (FSSF) to realize robust, real-time object tracking in hyperspectral surveillance video. Traditional object tracking in surveillance video based only on appearance information often fails in the presence of background clutter, low resolution, and appearance changes. Hyperspectral imaging uses unique spectral properties as well as spatial information to improve tracking accuracy in such challenging environments. However, the high-dimensionality of hyperspectral images causes high computational costs and difficulties for discriminative feature extraction. In FSSF, the real-time spatial-spectral convolution (RSSC) kernel is updated in real time in the Fourier transform domain without offline training to quickly extract discriminative spatial-spectral features. The spatial-spectral features are integrated into correlation filters to complete the hyperspectral tracking. To validate the proposed scheme, we collected a hyperspectral surveillance video (HSSV) dataset consisting of 70 sequences in 25 bands. Extensive experiments confirm the advantages and the efficiency of the proposed FSSF for object tracking in hyperspectral video tracking in challenging conditions of background clutter, low resolution, and appearance changes.

Rotation adaptive correlation filter for moving object tracking in satellite videos

As a new method of Earth observation, video satellite can provide high-temporal resolution remote sensing images for object tracking. Object tracking in satellite videos is promising yet challenging in computer vision. Although many algorithms for satellite video object tracking have been proposed, none of them solve the problem of tracking rotating object. Due to the nadir view, the rotation of an object is very common in the satellite videos. This problem urgently needs to be addressed. In this paper, a rotation-adaptive correlation filter (RACF) tracking algorithm is proposed to address the problem caused by the rotation of object. The proposed algorithm provides the following improvements: (a) A method of estimating the object rotation angle to keep the feature map stable during the object rotation is proposed. This method can overcome the drawback of histogram of oriented gradient (HOG) based trackers, which cannot deal with the rotation of objects in satellite videos; and (b) making the algorithm capable of estimating the change in the bounding box size caused by object’s rotation. The experimental results demonstrate that our algorithm can track object with a 99.84% precision score and 92.96% success score in six videos from the Jilin-1 satellite constellation.

A Survey on Object Detection and Tracking in Soccer Videos

"The players and ball are the most important object in soccer game videos and detected them are a challenging task because of many difficulties, such as shadow and illumination, ball size, several other objects look like a ball, often the ball overlapping with players or merged with lines, as well as the ball may be disappear which be hidden in the stadium or flying on air, and similar appearance of players, etc. The detect ball is the first step for tracking in broadcast soccer video. There are several methods of ball-tracking are based on their problem. In this paper, we have discussed different methods of object detection and tracking in the soccer videos which are available in the literature.

RETRACTED: Target detection algorithm for basketball moving images based on improved gaussian mixture model

The various methods used to detect and monitor basketball are explored in detail. Object detection and tracking and video Object detection and tracking are realized in a high-performance proportional manner. But most methods suffer from difficult predictions. The problematic reduction can occur at any stage, such as preprocessing, disassembly, feature extraction, background subtraction, and hole-filled ball tracking. In this article, the ball detection and tracking method based on the study method path is the growing method in the region, the public method, and the average transformation method using HSV color space and system features. Extensive inquiries into strategies, implementation issues, and future development trends. This research, edge detection, grayscale processing, object capture, target recognition, image detection technology, etc. are integrated into the game video's actual need to achieve the various requirements of motion image detection. At the same time, it has established a testing ground for research players, sports recognition, sports behavior judgment, etc. Background subtraction is a typical computer vision for Jobs. Methods examined Pixel is commonly used. Develop practical adaptive algorithms. Use a Gaussian probability density mixture. The recursive formula is used. Choose the appropriate number of parameters that are continuously updated and regions Per Pixel. The image data normalization and space-based communication planning and coordination of the work authorization model of small l is the static representation.

3-D Relation Network for visual relation recognition in videos

Video visual relation recognition aims at mining the dynamic relation instances between objects in the form of 〈subject,predicate,object〉, such as “person1-towards-person2” and “person-ride-bicycle”. Existing solutions treat the problem as several independent sub-tasks, i.e., image object detection, video object tracking and trajectory-based relation prediction. We argue that such separation results in the lack of information flow between different sub-models, which creates redundant representation while each sub-task cannot share a common set of task-specific features. Toward this end, we connect these three sub-tasks in an end-to-end manner by proposing the 3-D relation proposal that serves as a bridge for relation feature learning. Specifically, we put forward a novel deep neural network, named 3DRN, to fuse the spatio-temporal visual characteristics, object label features, and spatial interactive features for learning the relation instances with multi-modal cues. In addition, a three-staged training strategy is also provided to facilitate large-scale parameter optimization. We conduct extensive experiments on two public datasets with different emphasis to demonstrate the effectiveness of the proposed end-to-end feature learning method for visual relation recognition in videos. Furthermore, we verify the potential of our approach by tackling the video relation detection task.

Object Tracking in Satellite Videos Based on a Lightweight Network

Object tracking is a hot topic in computer vision. The significantly developed satellite video technology makes object tracking in satellite videos possible. In recent years, Convolutional Neural Network (CNN)-based trackers have achieved satisfying performance in the visual object tracking field. However, CNN cannot be directly applied to object tracking in satellite videos due to the following two reasons. First, the feature map size generally decreases as the network layer deepens, which is unsuitable for the small targets in satellite videos. Second, CNN-based trackers commonly need extensive data to train the network parameters, while few labeled satellite videos are available. Therefore, in this paper, we design a lightweight network for the satellite video tracking task. On one hand, the network generates a response map with the same size as the input image and reserves the spatial resolution of targets. On the other hand, the network parameters are transferred from an existing network and trained with the initial annotated frame, thus no extra data are needed. To make a fair comparison between existing trackers, we further propose a simulated benchmark based on the UAV123 dataset according to the imaging characteristics of satellite videos. Experiments are conducted to compare our method with other state-of-the-art trackers on both the simulated benchmark and real satellite videos and experimental results demonstrate the superiority of our proposed algorithm.

Enhanced TLD-based video object-tracking implementation tested on embedded platforms

Object-tracking algorithms on embedded platforms are very important in many civilian and military applications. The Tracking–Learning–Detection (TLD) algorithm is considered one of the state-of-the-art online long-term object-tracking algorithms. The performance of running such computationally intensive algorithms on embedded platforms with limited computing resources is a challenge. This work proposes an enhanced TLD implementation, specifically designed for, and tested on, embedded platforms. In this new implementation, an extra-stage has been added to the TLD detector cascade, called a Region filter. This filter dynamically identifies the candidate region for the tracked object. Further, the two independent tracker and detector TLD components, and the two independent Forward–Backward (FB) and Normalized Cross Correlation (NCC) error measures in the tracker have been parallelized. Still further, the computations of Image Integral in the detector and the NCC in both the tracker and the detector have been optimized using a single instruction multiple data (SIMD) architecture. We evaluate our proposed implementation on the Apalis T30 embedded platform, using the same video sequences that the original TLD is evaluated on. Our results show that our enhanced implementation outperforms the baseline with an average speedup of 3.7 × in the total number of frames per second (fps), while achieving an average 91% of the Precision and 86.6% of the Recall metrics, across all sequences. Further, our enhanced implementation achieves an average speedup of 4.52 × and 1.86 × in the detector and tracker execution times, respectively. Moreover, it results in an average 66.3% energy saving, as compared to the original implementation.