Cues For Tracking Research Articles

TrafficTrack: rethinking the motion and appearance cue for multi-vehicle tracking in traffic monitoring

TrafficTrack: rethinking the motion and appearance cue for multi-vehicle tracking in traffic monitoring

Modeling of Multiple Spatial-Temporal Relations for Robust Visual Object Tracking.

Recently, one-stream trackers have achieved parallel feature extraction and relation modeling through the exploitation of Transformer-based architectures. This design greatly improves the performance of trackers. However, as one-stream trackers often overlook crucial tracking cues beyond the template, they prone to give unsatisfactory results against complex tracking scenarios. To tackle these challenges, we propose a multi-cue single-stream tracker, dubbed MCTrack here, which seamlessly integrates template information, historical trajectory, historical frame, and the search region for synchronized feature extraction and relation modeling. To achieve this, we employ two types of encoders to convert the template, historical frames, search region, and historical trajectory into tokens, which are then collectively fed into a Transformer architecture. To distill temporal and spatial cues, we introduce a novel adaptive update mechanism, which incorporates a thresholding component and a local multi-peak component to filter out less accurate and overly disturbed tracking cues. Empirically, MCTrack achieves leading performance on mainstream benchmark datasets, surpassing the most advanced SeqTrack by 2.0% in terms of the AO metric on GOT-10k. The code is available at https://github.com/wsumel/MCTrack.

Unifying Motion and Appearance Cues for Visual Tracking via Shared Queries

Unifying Motion and Appearance Cues for Visual Tracking via Shared Queries

Exploring Multi-modal Spatial-Temporal Contexts for High-performance RGB-T Tracking.

In RGB-T tracking, there exist rich spatial relationships between the target and backgrounds within multi-modal data as well as sound consistencies of spatial relationships among successive frames, which are crucial for boosting the tracking performance. However, most existing RGB-T trackers overlook such multi-modal spatial relationships and temporal consistencies within RGB-T videos, hindering them from robust tracking and practical applications in complex scenarios. In this paper, we propose a novel Multi-modal Spatial-Temporal Context (MMSTC) network for RGB-T tracking, which employs a Transformer architecture for the construction of reliable multi-modal spatial context information and the effective propagation of temporal context information. Specifically, a Multi-modal Transformer Encoder (MMTE) is designed to achieve the encoding of reliable multi-modal spatial contexts as well as the fusion of multi-modal features. Furthermore, a Quality-aware Transformer Decoder (QATD) is proposed to effectively propagate the tracking cues from historical frames to the current frame, which facilitates the object searching process. Moreover, the proposed MMSTC network can be easily extended to various tracking frameworks. New state-of-the-art results on five prevalent RGB-T tracking benchmarks demonstrate the superiorities of our proposed trackers over existing ones.

Leveraging Local and Global Cues for Visual Tracking via Parallel Interaction Network

Despite that both local and context information are crucial for robust tracking, existing CNN-based and transformer-based methods mainly focus on one of these aspects. Consequently, the former fails to exploit rich global context information due to the limited receptive field, while the latter suffers from the deficiencies in constructing the local relationship among neighboring regions. To address this issue, we propose the SiamPIN tracker, based on our Parallel Interaction Network. It consists of two effective modules, namely Global Aggregation Block (GAB) and Local Process Block (LPB). GAB perceives the global context to capture the long-range spatial dependency through a transformer-based architecture. Meanwhile, LPB performs local information extraction using a CNN model to retain the detailed appearance information of the target. These two modules are connected consecutively to compose a Trans-Conv unit block, which transmits the global context information to the local feature extraction procedure, hence enables the interaction of global-local information flow. Several such blocks are cascaded so that our model can learn to aggregate local and context information interactively. The proposed tracker achieves state-of-the-art performance on six benchmark datasets, while maintaining a real time running speed.

GCEVT: Learning Global Context Embedding for Vehicle Tracking in Unmanned Aerial Vehicle Videos

Vehicle tracking in the unmanned aerial vehicle (UAV) videos is a fundamental but vital computer vision task. It mainly consists of two key components, that is, detection and reidentification (ReID). Recently, one-shot trackers, which integrate detection and ReID in a unified network, have received significant attention for their fast-tracking speed. However, existing one-shot trackers typically utilize local information to distinguish the detected targets. Due to the lack of global relations, which are key cues for tracking, these methods struggle to identify targets in UAV videos accurately. To alleviate the above issue, we deliberately design an ReID head that combines nonlocal blocks and the transformer layer to capture the global semantic relation in this letter. First, we propose a novel pyramid fusion network (PFN) to obtain the pixel-wise relations of features at multiple levels and aggregate them into features with richer semantic information. Then, we present a channel-wise transformer enhancer (CTE) to model the dependencies among the channels of the feature map and predict fine-grained identity embeddings. Extensive experiments on VisDrone2021 and UAVDT benchmarks demonstrate that our tracker, namely global context embedding for vehicle tracking (GCEVT), achieves state-of-the-art tracking performance.

Ownership and willingness to compete for resources.

Boyer proposes that ownership intuitions depend on tracking cues predictive of agents' motivations to compete for resources. However, the account may mis-predict people's intuitions about ownership, and it may also be too cognitively costly to be feasible. Even so, alternative accounts could benefit by taking inspiration from how the account handles thorny issues in the psychology of ownership.

Multi-domain collaborative feature representation for robust visual object tracking

Jointly exploiting multiple different yet complementary domain information has been proven to be an effective way to perform robust object tracking. This paper focuses on effectively representing and utilizing complementary features from the frame domain and event domain for boosting object tracking performance in challenge scenarios. Specifically, we propose common features extractor to learn potential common representations from the RGB domain and event domain. For learning the unique features of the two domains, we utilize a unique extractor for event based on Spiking neural networks to extract edge cues in the event domain which may be missed in RGB in some challenging conditions, and a unique extractor for RGB based on deep convolutional neural networks to extract texture and semantic information in RGB domain. Extensive experiments on standard RGB benchmark and real event tracking dataset demonstrate the effectiveness of the proposed approach. We show our approach outperforms all compared state-of-the-art tracking algorithms and verify event-based data is a powerful cue for tracking in challenging scenes.

Occlusion-aware Visual Tracker using Spatial Structural Information and Dominant Features

To overcome the problem of occlusion in visual tracking, this paper proposes an occlusion-aware tracking algorithm. The proposed algorithm divides the object into discrete image patches according to the pixel distribution of the object by means of clustering. To avoid the drifting of the tracker to false targets, the proposed algorithm extracts the dominant features, such as color histogram or histogram of oriented gradient orientation, from these image patches, and uses them as cues for tracking. To enhance the robustness of the tracker, the proposed algorithm employs an implicit spatial structure between these patches as another cue for tracking; Afterwards, the proposed algorithm incorporates these components into the particle filter framework, which results in a robust and precise tracker. Experimental results on color image sequences with different resolutions show that the proposed tracker outperforms the comparison algorithms on handling occlusion in visual tracking

Near-Online Multi-Pedestrian Tracking via Combining Multiple Consistent Appearance Cues

An important cue for multi-pedestrian tracking in video is the consistent appearance of an individual for quite a while. In this paper, we address multi-pedestrian tracking by learning a robust appearance model from the paradigm of tracking by detection. To separate detections of different pedestrians while assembling detections of the same pedestrian, we take advantage of the cue of consistent appearance and exploit three types of evidence from the recent, past and near-future. Existing online approaches only exploit the detection-to-detection and sequence-to-detection metrics, which focus on the recent and past appearance patterns respectively, while the future pedestrian appearance is simply ignored. This drawback is remedied in this paper by further considering the sequence-to-sequence metric, which resorts to near-future appearance presentation. Adaptive combination weights are learned to fuse these three different metrics. Moreover, we propose a novel Focal Triplet Loss to make the model focus more on hard examples than the easy ones. We demonstrate that this can significantly enhance the discriminating power of the model compared with treating every sample equally. Effectiveness and efficiency of the proposed method is verified by conducting comprehensive ablation studies and comparing with many competitive (offline/online/near-online) counterparts on the MOT16 and MOT17 Challenges.

Sensory Cues Modulate Smooth Pursuit and Active Sensing Movements.

Animals routinely use autogenous movement to regulate the information encoded by their sensory systems. Weakly electric fish use fore–aft movements to regulate visual and electrosensory feedback as they maintain position within a moving refuge. During refuge tracking, fish produce two categories of movements: smooth pursuit that is approximately linear in its relation to the movement of the refuge and ancillary active sensing movements that are nonlinear. We identified four categories of nonlinear movements which we termed scanning, wiggle, drift, and reset. To examine the relations between sensory cues and production of both linear smooth pursuit and nonlinear active sensing movements, we altered visual and electrosensory cues for refuge tracking and measured the fore–aft movements of the fish. Specifically, we altered the length and structure of the refuge and performed experiments with light and in complete darkness. Linear measures of tracking performance were better for shorter refuges (less than a body length) than longer ones (>1.5 body lengths). The magnitude of nonlinear active sensing movements was strongly modulated by light cues but also increased as a function of both longer refuge length and decreased features. Specifically, fish shifted swimming movements from smooth pursuit to scanning when tracking in dark conditions. Finally, fish appear to use nonlinear movements as an alternate tracking strategy in longer refuges: the fish may use more drifts and resets to avoid exiting the ends of the refuge.

Structural Constraint Data Association for Online Multi-object Tracking

Online two-dimensional (2D) multi-object tracking (MOT) is a challenging task when the objects of interest have similar appearances. In that case, the motion of objects is another helpful cue for tracking and discriminating multiple objects. However, when using a single moving camera for online 2D MOT, observable motion cues are contaminated by global camera movements and, thus, are not always predictable. To deal with unexpected camera motion, we propose a new data association method that effectively exploits structural constraints in the presence of large camera motion. In addition, to reduce incorrect associations with mis-detections and false positives, we develop a novel event aggregation method to integrate assignment costs computed by structural constraints. We also utilize structural constraints to track missing objects when they are re-detected again. By doing this, identities of the missing objects can be retained continuously. Experimental results validated the effectiveness of the proposed data association algorithm under unexpected camera motions. In addition, tracking results on a large number of benchmark datasets demonstrated that the proposed MOT algorithm performs robustly and favorably against various online methods in terms of several quantitative metrics, and that its performance is comparable to offline methods.

Visual tracking via Boolean map representations

In this paper, we present a simple yet effective Boolean map based representation that exploits connectivity cues for visual tracking. We describe a target object with histogram of oriented gradients and raw color features, of which each one is characterized by a set of Boolean maps generated by uniformly thresholding their values. The Boolean maps effectively encode multi-scale connectivity cues of the target with different granularities. The fine-grained Boolean maps capture spatially structural details that are effective for precise target localization while the coarse-grained ones encode global shape information that are robust to large target appearance variations. Finally, all the Boolean maps form together a robust representation that can be approximated by an explicit feature map of the intersection kernel, which is fed into a logistic regression classifier with online update, and the target location is estimated within a particle filter framework. The proposed representation scheme is computationally efficient and facilitates achieving favorable performance in terms of accuracy and robustness against the state-of-the-art tracking methods on the OTB50 and VOT2016 benchmark datasets.

Integrating Social Grouping for Multitarget Tracking Across Cameras in a CRF Model

Tracking multiple targets across nonoverlapping cameras aims at estimating the trajectories of all targets, and maintaining their identity labels consistent while they move from one camera to another. Matching targets from different cameras can be very challenging, as there might be significant appearance variation and the blind area between cameras makes the target’s motion less predictable. Unlike most of the existing methods that only focus on modeling the appearance and spatiotemporal cues for inter-camera tracking, this paper presents a novel online learning approach that considers integrating high-level contextual information into the tracking system. The tracking problem is formulated using an online learned conditional random field (CRF) model that minimizes a global energy cost. Besides low-level information, social grouping behavior is explored in order to maintain targets’ identities as they move across cameras. In the proposed method, pairwise grouping behavior of targets is first learned within each camera. During inter-camera tracking, track associations that maintain single camera grouping consistencies are preferred. In addition, we introduce an iterative algorithm to find a good solution for the CRF model. Comparison experiments on several challenging real-world multicamera video sequences show that the proposed method is effective and outperforms the state-of-the-art approaches.

Tracklet Association by Online Target-Specific Metric Learning and Coherent Dynamics Estimation.

In this paper, we present a novel method based on online target-specific metric learning and coherent dynamics estimation for tracklet (track fragment) association by network flow optimization in long-term multi-person tracking. Our proposed framework aims to exploit appearance and motion cues to prevent identity switches during tracking and to recover missed detections. Furthermore, target-specific metrics (appearance cue) and motion dynamics (motion cue) are proposed to be learned and estimated online, i.e., during the tracking process. Our approach is effective even when such cues fail to identify or follow the target due to occlusions or object-to-object interactions. We also propose to learn the weights of these two tracking cues to handle the difficult situations, such as severe occlusions and object-to-object interactions effectively. Our method has been validated on several public datasets and the experimental results show that it outperforms several state-of-the-art tracking methods.

Multi-object Tracking in Aerial Image Sequences using Aerial Tracking Learning and Detection Algorithm

<p>Vison based tracking in aerial images has its own significance in the areas of both civil and defense applications. A novel algorithm called aerial tracking learning detection which works on the basis of the popular tracking learning detection algorithm to effectively track single and multiple objects in aerial images is proposed in this study. Tracking learning detection (TLD) considers both appearance and motion features for tracking. It can handle occlusion to certain extent, and can work well on long duration video sequences. However, when objects are tracked in aerial images taken from platforms like unmanned air vehicle, the problems of frequent pose change, scale and illumination variations arise adding to low resolution, noise and jitter introduced by motion of the camera. The proposed algorithm incorporates compensation for the camera movement, algorithmic modifications in combining appearance and motion cues for detection and tracking of multiple objects and enhancements in the form of inter object distance measure for improved performance of the tracker when there are many identical objects in proximity. This algorithm has been tested on a large number of aerial sequences including benchmark videos, TLD dataset and many classified unmanned air vehicle sequences and has shown better performance in comparison to TLD.</p><p> </p>

Depth-assisted rectification for real-time object detection and pose estimation

RGB-D sensors have become in recent years a product of easy access to general users. They provide both a color image and a depth image of the scene and, besides being used for object modeling, they can also offer important cues for object detection and tracking in real time. In this context, the work presented in this paper investigates the use of consumer RGB-D sensors for object detection and pose estimation from natural features. Two methods based on depth-assisted rectification are proposed, which transform features extracted from the color image to a canonical view using depth data in order to obtain a representation invariant to rotation, scale and perspective distortions. While one method is suitable for textured objects, either planar or non-planar, the other method focuses on texture-less planar objects. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than some existing methods for object detection and pose estimation, especially when dealing with oblique poses.

시공간 영상분할을 이용한 이동 및 이동 중 정지물체 검출

본 논문에서는 이동 카메라 환경에서 이동 및 이동 중 정지물체를 검출하기 위한 방법을 제안한다. 이동 중에 일시적으로 정지한 물체는 검출 결과의 응용관점에서 볼 때 이동물체의 검출만큼이나 중요한데, 기존의 이동물체 검출 방법들은 이들을 배경과 구분하지 못하는 한계를 갖는다. 이러한 문제를 해결하기 위해 제안하는 방법에서는 이동 가능성 큐, 위치 가능성 큐, 그리고 색 분포 유사성 큐를 정의하여 이동물체 검출 및 지속적인 추적에 이용한다. 그래프 컷 알고리즘은 세 개의 큐를 결합하여 시공간 영상분할을 수행함으로써 이동 및 이동 중 정지물체를 검출한다. 제안하는 방법은 이동물체 뿐 아니라 이동 중 정지물체에 대해서도 검출이 가능함을 실험을 통해 증명하였다. This paper proposes a method for detection of objects temporally stop moving in video sequences taken by a moving camera. Even though the consequence of missed detection of those objects could be catastrophic in terms of application level requirements, not much attention has been paid in conventional approaches. In the proposed method, we introduce cues for consistent detection and tracking of objects: motion potential, position potential, and color distribution similarity. Integration of the three cues in the graph-cut algorithm makes possible to detect objects that temporally stop moving and are newly appearing. Experiment results prove that the proposed method can not only detect moving objects but also track objects stop moving.

Learning to track multiple targets.

Monocular multiple-object tracking is a fundamental yet under-addressed computer vision problem. In this paper, we propose a novel learning framework for tracking multiple objects by detection. First, instead of heuristically defining a tracking algorithm, we learn that a discriminative structure prediction model from labeled video data captures the interdependence of multiple influence factors. Given the joint targets state from the last time step and the observation at the current frame, the joint targets state at the current time step can then be inferred by maximizing the joint probability score. Second, our detection results benefit from tracking cues. The traditional detection algorithms need a nonmaximal suppression postprocessing to select a subset from the total detection responses as the final output and a large number of selection mistakes are induced, especially under a congested circumstance. Our method integrates both detection and tracking cues. This integration helps to decrease the postprocessing mistake risk and to improve performance in tracking. Finally, we formulate the entire model training into a convex optimization problem and estimate its parameters using the cutting plane optimization. Experiments show that our method performs effectively in a large variety of scenarios, including pedestrian tracking in crowd scenes and vehicle tracking in congested traffic.

A probabilistic graph-based framework for plug-and-play multi-cue visual tracking.

In this paper, we propose a novel approach for integrating multiple tracking cues within a unified probabilistic graph-based Markov random fields (MRFs) representation. We show how to integrate temporal and spatial cues encoded by unary and pairwise probabilistic potentials. As the inference of such high-order MRF models is known to be NP-hard, we propose an efficient spectral relaxation-based inference scheme. The proposed scheme is exemplified by applying it to a mixture of five tracking cues, and is shown to be applicable to wider sets of cues. This paves the way for a modular plug-and-play tracking framework that can be easily adapted to diverse tracking scenarios. The proposed scheme is experimentally shown to compare favorably with contemporary state-of-the-art schemes, and provides accurate tracking results.