Visual Tracking Research Articles

Video tracking of single cells to identify clustering behavior

Cancer cell clustering is a critical factor in metastasis, with cells often believed to migrate in groups as they establish themselves in new environments. This study presents preliminary findings from an in vitro experiment, suggesting that co-culturing cells provides an effective method for observing this phenomenon, even though the cells are grown as monolayers. We introduce a novel single-cell tracking approach based on graph theory to identify clusters in PC3 cells cultivated in both monoculture and co-culture with PC12 cells, using 66-h time-lapse recordings. The initial step consists of defining “linked” pairs of PC3 cells, laying the foundation for the application of graph theory. We propose two alternative definitions for cell pairings. The first method, Method 1, defines cells as “linked” at a given time t if they are close together within a defined time period before and after t. A second potential alternative method, Method 2, pairs cells if there is an overlap between the convex hulls of their respective tracks during this time period. Pairing cells enables the application of graph theory for subsequent analysis. This framework represents a cell as a vertex (node) and a relation between two cells as an edge. An interconnected set of high-degree nodes (nodes with many connections or edges) forms a subgraph, or backbone, that defines a patch (cluster) of cells. All nodes connected to this backbone are part of the subgraph. The backbone of high-degree nodes functions as a partition (or cut) of the initial graph. Two consecutive clusters in the video are considered to share the same identity if the following cluster contains at least p = 75 % of the cells from the preceding cluster, and the mean positions of their cells are within △r = 75μm. PC3 cells grown in co-culture appear to form persistent clusters exceeding 10 cells after 40–50 h incubation following seeding. In contrast, PC3 cells cultured alone (mono-culture) did not exhibit this behavior. This approach is experimental and requires further validation with a broader dataset.

Framework for abnormal event detection and tracking based on effective sparse factorization strategy

The idea of tracking video objects has evolved to facilitate the area of surveillance systems. However, most current research efforts lie in speedy abnormal event detection and tracking of objects of interest tracking. However, the primary challenge is dealing with complex video structures' inherent redundancy. The existing research models for video tracking are more inclined towards improving accuracy. In contrast, the consideration of a more significant proportion of mobile object dynamics, e.g. abnormal events, in motion over the crowded video frame sequence is mainly overlooked, which is essential to study a specific movement pattern of the object of interest appearing in the frame sequence concerning the cost of computation factors. The study thereby introduces a unique strategy of speedy abnormal event detection and tracking, which facilitates video tracking to assess a specific pattern of object of interest movement over complex and crowded video scenes, considering a unique learning-based approach. The extensive simulation outcome further shows that the proposed tracking model accomplishes better tracking accuracy yet retains an optimized computation cost compared to the baseline studies. The computation of video tracking also accomplishes higher detection rates even in the challenging constraints of partial/complete occlusion, illumination variation and background clutter.

Understanding Occlusion and Temporomandibular Joint Function Using Deep Learning and Predictive Modeling.

Advancements in artificial intelligence (AI)-driven predictive modeling in dentistry are outpacing the clinical translation of research findings. Predictive modeling uses statistical methods to anticipate norms related to TMJ dynamics, complementing imaging modalities like cone beam computed tomography (CBCT) and magnetic resonance imaging (MRI). Deep learning, a subset of AI, helps quantify and analyze complex hierarchical relationships in occlusion and TMJ function. This narrative review explores the application of predictive modeling and deep learning to identify clinical trends and associations related to occlusion and TMJ function. Debates persist regarding best practices for managing occlusal factors in temporomandibular joint (TMJ) function analysis while interpreting and quantifying findings related to the TMJ and occlusion and mitigating biases remain challenging. Data generated from noninvasive chairside tools such as jaw trackers, video tracking, and 3D scanners with virtual articulators offer unique insights by predicting variations in dynamic jaw movement, TMJ, and occlusion. The predictions help us understand the highly individualized norms surrounding TMJ function that are often required to address temporomandibular disorders (TMDs) in general practice. Normal TMJ function, occlusion, and the appropriate management of TMDs are complex and continue to attract ongoing debate. This review examines how predictive modeling and artificial intelligence aid in understanding occlusion and TMJ function and provides insights into complex dental conditions such as TMDs that may improve diagnosis and treatment outcomes with noninvasive techniques.

Correlation-Embedded Transformer Tracking: A Single-Branch Framework.

Developing robust and discriminative appearance models has been a long-standing research challenge in visual object tracking. In the prevalent Siamese-based paradigm, the features extracted by the Siamese-like networks are often insufficient to model the tracked targets and distractor objects, thereby hindering them from being robust and discriminative simultaneously. While most Siamese trackers focus on designing robust correlation operations, we propose a novel single-branch tracking framework inspired by the transformer. Unlike the Siamese-like feature extraction, our tracker deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it can suppress non-target features, resulting in target-aware feature extraction. The output features can be directly used to predict target locations without additional correlation steps. Thus, we reformulate the two-branch Siamese tracking as a conceptually simple, fully transformer-based Single-Branch Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT baseline, we summarize many effective design principles and propose an improved tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a local modeling layer to enhance shallow-level features. A unified relation modeling is proposed to remove complex handcrafted layer pattern designs. SuperSBT is further improved by masked image modeling pre-training, integrating temporal modeling, and equipping with dedicated prediction heads. Thus, SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves superior results on eight VOT benchmarks.

Comparing Reports with Actual Missing Persons’ Footwear

Search and rescue resources conduct missions for tens of thousands of persons annually. Visual trackers constitute one of many resources involved in searches for missing persons. Original research compares report details with actual find characteristics of 428 missing persons’ footwear (or lack thereof) from 1970 to 2024. Footwear report details such as type, brand, color, and size proved correct for 78%, 87%, 88%, and 91%, respectively, when missing persons were found. Paradoxical undressing, voluntary and involuntary removal, and cognitive impairments contributed to 25% of all subjects being found unshod. Data forms and a database were created to collect misper footwear information. A case study illustrates the beneficial application of footwear investigations. This new research provides a unique foundational understanding of how footwear information can be applied to search missions. KEY WORDS: Search and Rescue, Tracking, Shoe, Boot, Barefoot, Unshod, Signcutting

Dynamic calibration and multi-view fusion of active vision tracking system for surgical instrument

Abstract In recent years, surgical instrument tracking has gained significant attention in the field of surgical navigation. An often-overlooked problem lies in the instrument’s irregular movement conflicting with the tracking system’s fixed viewpoint, leading to line-of-sight (LOS) occlusion, marker occlusion, and sub-optimal accuracy. The active vision tracking system (which actively adjusts the camera viewpoint) is applied to address this issue. In this paper, two algorithms of multi-view fusion and dynamic calibration are proposed in multi-camera active vision to allow each camera to adjust its viewpoint independently. These algorithms are integrated into the parallel computing of the frontend and backend to implement the positioning of the instrument from multiple perspectives and real-time calibration of multiple cameras. Simulations and experiments are carried out to test the accuracy and robustness of the proposed tracking system. Results reveal that our system achieves a closer trajectory (0.12 mm) to the ground truth compared to a stereo camera tracking system (0.32 mm). The proposed tracking system is able to keep track of the instruments within the positioning volume at different surgical phases, ensuring consistent navigation and improving positioning stability and accuracy.

Humans and great apes visually track event roles in similar ways.

Human language relies on a rich cognitive machinery, partially shared with other animals. One key mechanism, however, decomposing events into causally linked agent-patient roles, has remained elusive with no known animal equivalent. In humans, agent-patient relations in event cognition drive how languages are processed neurally and expressions structured syntactically. We compared visual event tracking between humans and great apes, using stimuli that would elicit causal processing in humans. After accounting for attention to background information, we found similar gaze patterns to agent-patient relations in all species, mostly alternating attention to agents and patients, presumably in order to learn the nature of the event, and occasionally privileging agents under specific conditions. Six-month-old infants, in contrast, did not follow agent-patient relations and attended mostly to background information. These findings raise the possibility that event role tracking, a cognitive foundation of syntax, has evolved long before language but requires time and experience to become ontogenetically available.

Multi-Target Tracking with Multiple Unmanned Aerial Vehicles Based on Information Fusion

In high-altitude scenarios, targets tend to occupy a small number of pixels within the UAV’s field of view, resulting in substantial errors when identity recognition is attempted based solely on appearance features during multi-UAV joint tracking. Existing methodologies typically propose projecting multi-view data onto a single plane and leveraging distance information for identity association; however, their accuracy remains low as they are contingent on one-dimensional target information. To address this limitation, this paper introduces the UAVST-HM (UAV Swarm Tracking in High-altitude scenarios for Multiple targets) model, specifically designed to handle the characteristics of targets in the field of view of multiple UAVs at high altitudes. Initially, we develop techniques for extracting targets’ appearance, geometric, and distribution features. Subsequently, adaptive weights, calculated based on the mean of the respective features, are devised to amalgamate these diverse features, thereby constructing a cost matrix for cross-view target identity matching. This matrix is processed through the Hungarian algorithm, and multi-view target identity association is ultimately achieved via threshold filtering. On the MDMT dataset, our method enhances the MDA indicator, which assesses cross-view target identity matching, by 1.78 percentage points compared to the current state of the art. This significant enhancement substantially improves the overall efficacy of multi-UAV joint visual tracking from a high-altitude perspective.

UAV Visual Object Tracking Based on Spatio-Temporal Context

To balance the real-time and robustness of UAV visual tracking on a single CPU, this paper proposes an object tracker based on spatio-temporal context (STCT). STCT integrates the correlation filter and Siamese network into a unified framework and introduces the target’s motion model, enabling the tracker to adapt to target scale variations and effectively address challenges posed by rapid target motion, etc. Furthermore, a spatio-temporal regularization term based on the dynamic attention mechanism is proposed, and it is introduced into the correlation filter to suppress the aberrance of the response map. The filter solution is provided through the alternating direction method of multipliers (ADMM). In addition, to ensure efficiency, this paper proposes the average maximum response value-related energy (AMRE) for adaptive tracking state evaluation, which considers the time context of the tracking process in STCT. Experimental results show that the proposed STCT tracker can achieve a favorable balance between tracking robustness and real-time performance for UAV object tracking while running at ∼38 frames/s on a low-cost CPU.

Applicability of using dynamic MRI to evaluate alleged cranial rhythmic impulse (CRI)

ObjectivesTo evaluate the feasibility of using dynamic MRI to measure the features of cranial rhythmic impulse (CRI).Design and settingFifteen healthy participants (9 females and 6 males, aged 25 to 77) underwent dynamic MRI in a sagittal T2 HASTE view at a rate of 0.60 Hz for 30 s. The MRI videos were analyzed using video tracking software. Three points were marked: the glabella, the midpoint of the sella turcica, and a symmetrical point of the glabella on the occiput. The distances between these points were measured across 46 frames. Amplitudes and rates of asymmetrical CRI waves were calculated using Excel formulas.ResultsThe mean wave frequencies were 5.65 Hz for the anteroposterior distance, 6.2 Hz from sella turcica to occiput, and 6.76 Hz from sella turcica to glabella. The mean wave amplitudes were 0.39 mm, 0.6 mm, and 0.49 mm for the respective distances. Both intraclass correlation coefficients (ICC) and reliability coefficient (R) indicated excellent reliability (R, ICC > 0.90). The technical error of measurement (TEM) exceeded 1 mm for the anteroposterior and sella-to-occiput distances, while it was 0.32 mm for the sella-to-glabella distance.ConclusionsDynamic MRI demonstrates potential in measuring the features of CRI, particularly in assessing CRI wave rate. While the ICC values indicate high reliability, the TEM values suggest that using MRI to measure CRI wave amplitude may only be dependable for the distance from the sella to the glabella.

Research and Design of an Active Light Source System for UAVs Based on Light Intensity Matching Model

The saliency feature is a key factor in achieving vision-based tracking for multi-UAV control. However, due to the complex and variable environments encountered during multi-UAV operations—such as changes in lighting conditions and scale variations—the UAV’s visual features may degrade, especially under high-speed movement, ultimately resulting in failure of the vision tracking task and reducing the stability and robustness of swarm flight. Therefore, this paper proposes an adaptive active light source system based on light intensity matching to address the issue of visual feature loss caused by environmental light intensity and scale variations in multi-UAV collaborative navigation. The system consists of three components: an environment sensing and control module, a variable active light source module, and a light source power module. This paper first designs the overall framework of the active light source system, detailing the functions of each module and their collaborative working principles. Furthermore, optimization experiments are conducted on the variable active light source module. By comparing the recognition effects of the variable active light source module under different parameters, the best configuration is selected. In addition, to improve the robustness of the active light source system under different lighting conditions, this paper also constructs a light source color matching model based on light intensity matching. By collecting and comparing visible light images of different color light sources under various intensities and constructing the light intensity matching model using the comprehensive peak signal-to-noise ratio parameter, the model is optimized to ensure the best vision tracking performance under different lighting conditions. Finally, to validate the effectiveness of the proposed active light source system, quantitative and qualitative recognition comparison experiments were conducted in eight different scenarios with UAVs equipped with active light sources. The experimental results show that the UAV equipped with an active light source has improved the recall of yoloV7 and RT-DETR recognition algorithms by 30% and 29.6%, the mAP50 by 21% and 19.5%, and the recognition accuracy by 13.1% and 13.6, respectively. Qualitative experiments also demonstrated that the active light source effectively improved the recognition success rate under low lighting conditions. Extensive qualitative and quantitative experiments confirm that the UAV active light source system based on light intensity matching proposed in this paper effectively enhances the effectiveness and robustness of vision-based tracking for multi-UAVs, particularly in complex and variable environments. This research provides an efficient and computationally effective solution for vision-based multi-UAV systems, further enhancing the visual tracking capabilities of multi-UAVs under complex conditions.

FETrack: Feature-Enhanced Transformer Network for Visual Object Tracking

Visual object tracking is a fundamental task in computer vision, with applications ranging from video surveillance to autonomous driving. Despite recent advances in transformer-based one-stream trackers, unrestricted feature interactions between the template and the search region often introduce background noise into the template, degrading the tracking performance. To address this issue, we propose FETrack, a feature-enhanced transformer-based network for visual object tracking. Specifically, we incorporate an independent template stream in the encoder of the one-stream tracker to acquire the high-quality template features while suppressing the harmful background noise effectively. Then, we employ a sequence-learning-based causal transformer in the decoder to generate the bounding box autoregressively, simplifying the prediction head network. Further, we present a dynamic threshold-based online template-updating strategy and a template-filtering approach to boost tracking robustness and reduce redundant computations. Extensive experiments demonstrate that our FETrack achieves a superior performance over state-of-the-art trackers. Specifically, the proposed FETrack achieves a 75.1% AO on GOT-10k, 81.2% AUC on LaSOT, and 89.3% Pnorm on TrackingNet.

F-measure and Intra-operator Reliability of Aerial Imagery Tracking in Football

Current player tracking methods using multiple fixed stadium-based cameras and wearable sensors have limitations. To address this, a new computer vision system using aerial drone imagery has been developed to track football players. This approach is less expensive, has a wider field of view, and captures data not accessible through sensors. However, the visual performance has not been extensively evaluated. In this study, we aimed to determine the system’s tracking performance using an F-measure score, which is calculated based on the number of true positives, false positives, and false negatives identified during the tracking process. We also investigated the tracking reliability by comparing the intra-operator performance using the ICC for distance, speed, and time metrics by repeating the measurements five times. The aerial-imagery data were taken from a test match recorded using a drone that was hovered away from the touchline. Four players were tracked and measured simultaneously. The system demonstrates accuracy by performing admirably with average F-measure scores of 0.80, 0.80, 0.89, and 0.84 for player A0, player A1, player B0, and player B1, respectively. Meanwhile, the intra-operator reliability for distance and speed was deemed good to moderate with %MD < 10%. The findings suggest that the system is a capable and reliable computer vision tracking tool with potential applications in performance analysis, training feedback, and injury prevention.

Path Planning and Motion Control of Robot Dog Through Rough Terrain Based on Vision Navigation.

This article delineates the enhancement of an autonomous navigation and obstacle avoidance system for a quadruped robot dog. Part one of this paper presents the integration of a sophisticated multi-level dynamic control framework, utilizing Model Predictive Control (MPC) and Whole-Body Control (WBC) from MIT Cheetah. The system employs an Intel RealSense D435i depth camera for depth vision-based navigation, which enables high-fidelity 3D environmental mapping and real-time path planning. A significant innovation is the customization of the EGO-Planner to optimize trajectory planning in dynamically changing terrains, coupled with the implementation of a multi-body dynamics model that significantly improves the robot's stability and maneuverability across various surfaces. The experimental results show that the RGB-D system exhibits superior velocity stability and trajectory accuracy to the SLAM system, with a 20% reduction in the cumulative velocity error and a 10% improvement in path tracking precision. The experimental results also show that the RGB-D system achieves smoother navigation, requiring 15% fewer iterations for path planning, and a 30% faster success rate recovery in challenging environments. The successful application of these technologies in simulated urban disaster scenarios suggests promising future applications in emergency response and complex urban environments. Part two of this paper presents the development of a robust path planning algorithm for a robot dog on a rough terrain based on attached binocular vision navigation. We use a commercial-of-the-shelf (COTS) robot dog. An optical CCD binocular vision dynamic tracking system is used to provide environment information. Likewise, the pose and posture of the robot dog are obtained from the robot's own sensors, and a kinematics model is established. Then, a binocular vision tracking method is developed to determine the optimal path, provide a proposal (commands to actuators) of the position and posture of the bionic robot, and achieve stable motion on tough terrains. The terrain is assumed to be a gentle uneven terrain to begin with and subsequently proceeds to a more rough surface. This work consists of four steps: (1) pose and position data are acquired from the robot dog's own inertial sensors, (2) terrain and environment information is input from onboard cameras, (3) information is fused (integrated), and (4) path planning and motion control proposals are made. Ultimately, this work provides a robust framework for future developments in the vision-based navigation and control of quadruped robots, offering potential solutions for navigating complex and dynamic terrains.

Improving Visual Object Detection using General UFIR Filtering

Object detection is a fundamental task in computer vision, which involves the identification and localization of objects within image frames or video sequences. The problem is complicated by large variations in the video camera bounding box, which can be thought of as colored measurement noise (CMN). In this paper, we use the general unbiased finite impulse response (GUFIR) approach to improve detection performance under CMN. The results are compared to the general Kalman filter (GKF) and two detection methods: “Faster-RCNN” and “Tensorflow PASCAL Visual Object Classes (VOC)”. Experimental testing is carried out using the benchmark data ”Car4”. It is shown that GUFIR significantly improves the detection accuracy and demonstrates the properties of the effective tool for visual object tracking.

Efficacy Of Visual Tracking Exercises Among Stroke Survivors

Efficacy Of Visual Tracking Exercises Among Stroke Survivors

Advancements in deep learning for visual object tracking

Abstract. Since contemporary information-retrieval systems rely heavily on the content of titles and abstracts to identify relevant articles in literature searches, great care should be taken in constructing both. This comprehensive review delves into the transformative impact of deep learning on the domain of visual object tracking. Since the inception of AlexNet in 2012, deep learning has revolutionized feature extraction, leading to significant advancements in tracking accuracy and robustness. The article explores the integration of deep learning with various tracking algorithms, including deep correlation filters, classification-based approaches, Siamese networks, gradient-based methods, and the innovative application of Transformer architectures. Moreover, the role of tracking datasets in fostering algorithm development and innovation is highlighted, with an emphasis on the expansion in scale, diversity, and annotation quality. Furthermore, the article also examines the multifaceted evaluation metrics for tracking algorithms, encompassing precision, robustness, efficiency, generalization, and real-time capabilities. Looking ahead, the review outlines future research directions, such as algorithm optimization for lightweight and accelerated performance, enhancing generalizability, leveraging multimodal data fusion, and refining Transformer models for improved temporal information processing. The challenges of long-term tracking and the growing importance of algorithm interpretability and transparency are also discussed. In summary, the article underscores the promising trajectory of deep learning in visual object tracking, with ongoing research poised to make tracking technologies smarter, more efficient, and robust, catering to a wide array of practical applications and environments.

Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging

Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging

CTIFTrack: Continuous Temporal Information Fusion for object track

In visual tracking tasks, researchers usually focus on increasing the complexity of the model or only discretely focusing on the changes in the object itself to achieve accurate recognition and tracking of the moving object. However, they often overlook the significant contribution of video-level linear temporal information fusion and continuous spatiotemporal mapping to tracking tasks. This oversight may lead to poor tracking performance or insufficient real-time ability of the model in complex scenes. Therefore, this paper proposes a real-time tracker, namely Continuous Temporal Information Fusion Tracker (CTIFTrack). The key of CTIFTrack lies in its well-designed Temporal Information Fusion (TIF) module, which cleverly performs a linear fusion of the temporal information between the (t-1)-th and the t-th frames and completes the spatiotemporal mapping. This enables the tracker to better understand the overall spatiotemporal information and contextual spatiotemporal correlations within the video, thereby having a positive impact on the tracking task. In addition, this paper also proposes the Object Template Feature Refinement (OTFR) module, which effectively captures the global information and local details of the object, and further improves the tracker’s understanding of the object features. Extensive experiments are conducted on seven benchmarks, such as LaSOT, GOT-10K, UAV123, NFS, TrackingNet, VOT2018 and OTB-100. The experimental results validate the significant contribution of the TIF module and OTFR module to the tracking task, as well as the effectiveness of CTIFTrack. It is worth noting that while maintaining excellent tracking performance, CTIFTrack also shows outstanding real-time tracking speed. On the Nvidia Tesla T4-16GB GPU, the FPS of CTIFTrack reaches 71.98. The code and demo materials will be available at https://github.com/vpsg-research/CTIFTrack.

Improving CNN Fish Detection and Classification with Tracking

The regular and consistent monitoring of marine ecosystems and fish communities is becoming more and more crucial due to increasing human pressures. To this end, underwater camera technology has become a major tool to collect an important amount of marine data. As the size of the data collected outgrew the ability to process it, new means of automatic processing have been explored. Convolutional neural networks (CNNs) have been the most popular method for automatic underwater video analysis for the last few years. However, such algorithms are rather image-based and do not exploit the potential of video data. In this paper, we propose a method of coupling video tracking and CNN image analysis to perform a robust and accurate fish classification on deep sea videos and improve automatic classification accuracy. Our method fused CNNs and tracking methods, allowing us to detect 12% more individuals compared to CNN alone.