Visual Tracking Research Articles

Segment-to-track for pavement crack with light-weight neural network on unmanned wheeled robot

Instance segmentation of pavement crack presents notable challenges but is pivotal for practical applications such as crack visual tracking and automated crack repair. However, slender cracks are plagued by class imbalance, low tolerance for interference in unstructured road scenes, and the presence of topological features. In this article, a novel approach called S2TNet is proposed, which enhances the contrast between the crack image foreground and background using local contrast enhancement. Specifically, the proposed anchor ratio IoU-Balanced Sampling (ARIS) and Balanced Fine-Grained Features (BFGF) effectively improve the performance of the detector in predicting bounding box and segmenting instance binary mask. Moreover, we establish a framework using an unmanned wheeled robot with a four-wheel independent differential drive system to deploy the S2TNet approach. Extensive experiments conducted on self-created (S2T-Crack) datasets present the effectiveness of the proposed method in achieving real-time segment-to-track at a speed of 0.05 m/s, with a segmentation precision of 80.21% for pavement cracks.

Participatory hackathon to determine ecological relevant endpoints for a neurotoxin to aquatic and benthic invertebrates.

The aim of this study is twofold: i) to determine innovative yet sensitive endpoints for sulfoxaflor and ii) to develop best practices for innovative teaching in ecotoxicology. To this end, a group of 52 MSc students participated in an environmental hackathon, during which they did creative toxicity testing on 5 freshwater invertebrate species: Daphnia magna, Chironomus riparius, Asellus aquaticus, Lymnaea stagnalis, and Anisus vortex. Involving the students in an active learning environment stimulated increased creativity and productivity. In total, 28 endpoints were investigated, including standard endpoints (e.g., mortality) as well as biomechanistic and energy-related endpoints. Despite high variances in the results, likely linked to the limited lab experience of the students and interpersonal differences, a promising set of endpoints was selected for further investigation. A more targeted follow-up experiment focused on the most promising organism and set of endpoints: biomechanistic endpoints of C. riparius larvae. Larvae were exposed to a range of sulfoxaflor concentrations (0.90-67.2 μg/L) for 21 days. Video tracking showed that undulation and swimming were significantly reduced at 11.1 μg sulfoxaflor/L after 9 days of exposure, and an EC50 = 10.6 μg/L for mean velocities of the larvae in the water phase was found. Biomechanistic endpoints proved much more sensitive than mortality, for which an LC50 value of 116 μg/L was found on Day 9. Our results show that performing a hackathon with students has excellent potential to find sensitive endpoints that can subsequently be verified using more targeted and professional follow-up experiments. Furthermore, utilising hackathon events in teaching can increase students' enthusiasm about ecotoxicology, driving better learning experiences.

SiamFDA: feature dynamic activation siamese network for visual tracking.

In this paper, we present a novel anchor-free visual tracking framework, referred to as feature dynamic activation siamese network (SiamFDA), which addresses the issue of ignoring global spatial information in current Siamese network-based tracking algorithms. Our approach captures long-range dependencies between distant pixels in space, which enables robustness to unreliable regions. Additionally, we introduce a hierarchical feature selector that adaptively activates features at different layers, and an adaptive sample label assignment method to further improve tracking performance. Our extensive evaluations on six benchmark datasets, including VOT-2018, VOT-2019, GOT10k, LaSOT, OTB-2015, and OTB-2013, demonstrate that SiamFDA outperforms several state-of-the-art trackers in various challenging scenarios, with a real-time frame rate of 40 frames per second.

Effect of Al-Cu-Fe Quasicrystal Particles on the Reinforcement of a Polymer–Matrix Composite: From Surface to Mechanical Properties

We examined the effect of Al59Cu25Fe13B3 (at.%) quasicrystalline (QC) reinforcement particles on the mechanical and surface properties of a polymer-matrix composite by applying a technical polymer polyphthalamide (PPA). The observed increase in the tensile Young’s modulus ranged from 1810 MPa for the pure polymer to 4114 MPa for the composite with a QC filling of 35 vol.%. The elongation at fracture decreased with the filling fraction, being equal to 16.9% for a pure polymer and dropping to 4.8% for the composite with a QC filling of 35 vol.%. The same trend was noticeable with flexural Young’s modulus, which ranged from 100 MPa for a pure polymer to 125.5 MPa for the composite with 35 vol.% of QC. It was found that the increase in the mechanical strength led to a simultaneous increase of brittleness, which was reflected in a decrease of the impact strength for a pure polymer from 98.5 kJ/m2 to 42.4 kJ/m2 for composites with a QC filling of 35 vol.%. In contrast, when filled with 5 vol.% of QC, the impact strength increased by 8%. The friction coefficient against 100C6 steel dropped from 0.15 for pure PPA down to 0.10 for 5 vol.% of the QC filling, followed by an increase to 0.26 for further QC fillings up to 35 vol.%. Interestingly, a local minimum of friction was achieved at filling factors between 5 to 20 vol.% of QC. Independently, a clear surfenergy minimum was also found for the composite material with 20 vol.% of QC filling associated with a net drop in the polar component of the surfenergy. Surfenergy refers to the surface energy related to the top of the oxide layer under ambient conditions. We hypothesise that this is related to the percolation threshold at about 13 vol.% QC, reflected in the observed behaviour of both the friction coefficient and surfenergy. For the pure QC annealed in air for 1 h at 500 °C significant wear tracks were observed accompanied by a wear debris formation. On the other hand, a pure polymer exhibited slightly visible wear tracks with no apparent debris formation, and for the composites with different QC filling factors, the wear traces were barely visible with negligible debris formation.

Self-Prompting Tracking: A Fast and Efficient Tracking Pipeline for UAV Videos

In the realm of visual tracking, remote sensing videos captured by Unmanned Aerial Vehicles (UAVs) have seen significant advancements with wide applications. However, there remain challenges to conventional Transformer-based trackers in balancing tracking accuracy and inference speed. This problem is further exacerbated when Transformers are extensively implemented at larger model scales. To address this challenge, we present a fast and efficient UAV tracking framework, denoted as SiamPT, aiming to reduce the number of Transformer layers without losing the discriminative ability of the model. To realize it, we transfer the conventional prompting theories in multi-model tracking into UAV tracking, where a novel self-prompting method is proposed by utilizing the target’s inherent characteristics in the search branch to discriminate targets from the background. Specifically, a self-distribution strategy is introduced to capture feature-level relationships, which segment tokens into distinct smaller patches. Subsequently, salient tokens within the full attention map are identified as foreground targets, enabling the fusion of local region information. These fused tokens serve as prompters to enhance the identification of distractors, thereby avoiding the demand for model expansion. SiamPT has demonstrated impressive results on the UAV123 benchmark, achieving success and precision rates of 0.694 and 0.890 respectively, while maintaining an inference speed of 91.0 FPS.

Development of a 3D tracking system for multiple marmosets under free-moving conditions

Assessment of social interactions and behavioral changes in nonhuman primates is useful for understanding brain function changes during life events and pathogenesis of neurological diseases. The common marmoset (Callithrix jacchus), which lives in a nuclear family like humans, is a useful model, but longitudinal automated behavioral observation of multiple animals has not been achieved. Here, we developed a Full Monitoring and Animal Identification (FulMAI) system for longitudinal detection of three-dimensional (3D) trajectories of each individual in multiple marmosets under free-moving conditions by combining video tracking, Light Detection and Ranging, and deep learning. Using this system, identification of each animal was more than 97% accurate. Location preferences and inter-individual distance could be calculated, and deep learning could detect grooming behavior. The FulMAI system allows us to analyze the natural behavior of individuals in a family over their lifetime and understand how behavior changes due to life events together with other data.

Effect of intermittent subchronic MK-801 administration on dopamine synthesis capacity and responsiveness in the prefrontal cortex.

The therapeutic potential of N-methyl-D-aspartate glutamate receptor (NMDAR) antagonists, particularly ketamine, in mood disorders, is linked to their modulation of dopamine dynamics in the medial prefrontal cortex (mPFC). However, conflicting effects of distinct NMDAR antagonists, like ketamine and phencyclidine, on mPFC dopamine levels stem from variances in their receptor affinity profiles. This study investigates the impact of intermittent subchronic administration of an NMDAR antagonist on dopamine synthesis capacity and responsiveness within the mPFC, focusing on Dizocilpine (MK-801), a highly selective NMDAR antagonist. In vivo microdialysis and high-performance liquid chromatography assessed extracellular dopamine levels in the mPFC following subchronic MK-801 treatment. Locomotor activity was measured using a computed video tracking system. Intermittent subchronic MK-801 administration, followed by a 24-h withdrawal, preserved both dopamine synthesis capacity and responsiveness to MK-801 challenge in the mPFC. However, altered locomotor activity was observed, deviating from previous findings indicating impaired dopamine synthesis and responsiveness in the mPFC with twice-daily subchronic NMDAR antagonist treatment. These findings offer crucial biochemical insights into the diverse impacts of NMDAR antagonists on dopamine dynamics and the distinct therapeutic mechanisms associated with ketamine in depression treatment. However, further investigation is imperative to pinpoint potential inconsistencies stemming from variances in drug type, dosage, or administration frequency.

Multiple object detection and tracking from drone videos based on GM-YOLO and multi-tracker

Multiple object tracking in drone videos is a vital vision task with broad application prospects, but most trackers use spatial or appearance clues alone to correlate detections. Our proposed Multi-Tracker uses a novel similarity measure that combines position and appearance information. We designed the GM-YOLO network to provide high-quality detections as input to Multi-Tracker. Add a Coordinate Attention mechanism and a weighted Bidirectional Feature Pyramid Network structure to the Backbone, each feature point's effective receptive field is modeled as a Gaussian distribution. To accurately obtain the motion and appearance features of the object, the adaptive noise covariance Kalman filter is used to get the position information, MB-OSNet network is designed to use global features to learn contour information to retrieve images from a wider field of view while incorporating Part-Level elements that contain more fine-grained data. Finally, the motion and appearance features are jointly compared to realize multi object tracking. The performance of the GM-YOLO object detector and the Multi-Tracker was verified on the VisDrone MOT and UAVDT datasets.

Siamada: visual tracking based on Siamese adaptive learning network

Siamada: visual tracking based on Siamese adaptive learning network

Efficient Object Detection and Classification Approach Using an Enhanced Moving Object Detection Algorithm in Motion Videos

Object detection and classification have become prominent research topics in computer vision due to their applications in areas such as visual tracking. Despite advancements, vision-based methods for detecting smaller targets and densely packed objects with high accuracy in complex dynamic environments still encounter challenges. This paper introduces a novel and enhanced approach for hyperbolic shadow detection and object classification based on the Enhanced Moving Object Detection (EMOD) algorithm and an improved manta ray-based convolutional neural network optimized for search. In the preprocessing phase, the video data transforms into a sequence of frames, with polynomial adaptive antialiasing applied to maintain frame size and reduce noise. Additionally, an enhanced boundary area preservation algorithm improves the contrast of noise-free edited image sequences. To achieve high-precision detection of smaller objects, the Grib profile of each detected object is also tracked. Finally, a convolutional neural network method employing an enhanced Manta search optimization is deployed for target detection and classification. Comparative experiments conducted across diverse datasets and benchmark methods demonstrate significantly improved accuracy and expanded capabilities in detection and classification.

Exploiting multi-scale hierarchical feature representation for visual tracking

Convolutional neural networks (CNNs) have been the dominant architectures for feature extraction tasks, but CNNs do not look for and focus on some specific image features. Correlation operations play an important role in visual tracking. However, the correlation operation reserves a large amount of unfavorable background information. In this paper, we propose an effective feature recognizer including channel and spatial attention modules to focus on important object feature information. Thus, the representation power of the feature extraction network is improved. Further, we design a multi-scale feature fusion network. The fusion network performs feature fusion on template feature and encoded feature branches to establish connections between features at different scales. Experiments on six benchmarks demonstrate that the proposed tracker outperforms the state-of-the-art trackers. In particular, the proposed tracker achieves an 80.4% AUC on TrackingNet and a 68.4% AUC on GOT-10k while running at a real-time speed.

Adaptive Control of Underwater Tunnel Monitoring Robot Based on IoT and Fuzzy Neural Network Algorithm

Abstract To improve the navigation ability of underwater tunnel monitoring robots at fixed distances, directions, depths, and heights and to improve the accuracy of tunnel monitoring, an adaptive control method for underwater tunnel monitoring robots based on the Internet of Things (IoT) and fuzzy neural network algorithms is proposed. The structure of underwater tunnel monitoring robots is analyzed based on the IoT, the convolutional neural network algorithm is used to extract the tracking target characteristics of the underwater tunnel monitoring robot, and the obstacle avoidance process of the underwater tunnel monitoring robot is analyzed. The membership degree of the input variable is calculated by the fuzzy control algorithm. The control rule optimizes the neural network algorithm, obtains the target characteristics displayed by the visual tracking of the underwater tunnel monitoring robot based on the fuzzy neural network, uses the adaptive control to estimate the optimal parameters, and finally obtains the adaptive sliding mode control of the underwater tunnel monitoring robot. The experimental results show that the proposed method can accurately realize the target tracking task of the underwater tunnel monitoring robot and has better obstacle avoidance ability.

Exploring the potential of visual tracking and counting for trees infected with pine wilt disease based on improved YOLOv5 and StrongSORT algorithm

Pine wilt disease (PWD) has been consistently recognized as one of the most catastrophic forest diseases in China over the past four decades. Accurate identification and timely removal of infected pine trees are vital for controlling the disease spread. However, previous studies about the identification of PWD-infected trees still relied on traditional machine learning methods, with static imagery being the predominant data form utilized. Due to diverse forest environments, there are significant errors in wide-range identification and the collaborative adaptation capability between multiple algorithms is suboptimal. Real-time dynamic tracking and counting of PWD-infected trees based on deep learning have received little attention. Thus, an improved YOLOv5 was proposed in this study, which in synergy with StrongSORT, enables the tracking and counting of PWD-infected trees in a dynamic visual way. For this purpose, a dataset of 6,450 static images (39,809 PWD-infected tree samples) was constructed for model training and validation, and 130 dynamic video segments (approximately 210,000 frames) and 674 static images were used to evaluate the proposed method. To enhance feature extraction efficiency in deep learning networks, the Second-Order Channel Attention (SOCA) mechanism was introduced, and the Simplified Spatial Pyramid Pooling-Fast (SimSPPF) was employed as a replacement for the original SPPF. Additionally, for the geometric features of PWD-infected trees, a more scientific Weighted Boxes Fusion (WBF) strategy was utilized during the prediction phase to construct detection boxes, which contributes to better detection of dense targets. Regarding detection, the improved YOLOv5 performs optimally, with mAP@0.5 and F1-Score of 92.4 % and 88.3 %, respectively, an increase of 2.5 % and 1 % compared to the original model. The generalization capability has been evaluated on the test set, all metrics exceeded 90 %. In terms of tracking, the combination of the improved YOLOv5 with StrongSORT yields Identification F1 (IDF1), High-Order Tracking Accuracy (HOTA), Multi-Object Tracking Accuracy (MOTA), and Multi-Object Tracking Precision (MOTP) of 75.4 %, 55.6 %, 63.5 %, and 72.3 % respectively, showcasing increase of 3.5 %, 2.7 %, 6 %, and 0.3 % compared to the original model. Notably, the Mostly Lost (ML) and Identity Switches (IDSW) are reduced by 43 % and 20 % respectively. Concerning counting, the proposed method was evaluated on 130 dynamic video segments, indicating a high correlation with the Ground truth (R2 = 0.965), affirming its effectiveness. In summary, visual tracking and counting of PWD-infected trees in complex forest areas can be enabled by the method proposed, providing a new approach for the intelligent monitoring and management of PWD.

BAN-ima: A Box Adaptive Network With Iterative Mixed Attention for Visual Tracking

BAN-ima: A Box Adaptive Network With Iterative Mixed Attention for Visual Tracking

Online Learning Samples and Adaptive Recovery for Robust RGB-T Tracking

With the increasing diversity of visual tracking tasks, object tracking in RGB and thermal (RGB-T) modalities has received widespread interest. Most of the existing RGB-T tracking methods mainly improve tracking performance by integrating hierarchically complementary information from RGB and thermal modalities, however, they are insufficient in handling tracking failures due to the lack of re-detection capability. To address these issues, we propose a new RGB-T tracking method with online learning samples and adaptive object recovery. First, the features of RGB and thermal modalities are concatenated for robust appearance modeling. Second, a multimodal fusion strategy is designed to stably integrate reliable information of modalities and propose to use similarity to measure tracking confidence. Finally, a detector with online learning of positive and negative samples and adaptive recovery is developed to correct unreliable tracking results. Numerical results on five recent large-scale benchmark datasets demonstrate that the proposed tracker achieves competitive performance compared to other state-of-the-art methods.

Kinematics analysis on accelerated motion using tracker video analysis for educational purposes

This study aims to determine the effectiveness of video analysis tracker-based practicum on the material of the motion of objects on an inclined plane and the motion of objects in free fall. The use of video tracker analysis-based practicum is useful for determining some kinematics parameters of objects that cannot only be calculated by traditional practicum methods. By using the video tracker analysis method, all object kinematics parameters can be found directly in visual form. The results show that (1) There is no relationship between mass and kinematics parameters. (2) There is an influence due to air friction on the acceleration value of the object. (3) The suitability of the concept with the tracker application produced with the error value in the experiment of object motion on an inclined plane () (4) The suitability of the concept with the tracker application produced with the error value in the experiment of object motion in free fall () (5) The graphical representation of the tracker is in accordance with the theory studied. This research implies that the existence of learning using practicum, especially practicum-based video tracker analysis can improve understanding and process skills in understanding science concepts. This research is written so that the programme can be further applied to science education.

Evolution of Siamese Visual Tracking with Slot Attention

Siamese network object tracking is a widely employed tracking method due to its simplicity and effectiveness. It first employs a two-stream network to independently extract template and search region features. Subsequently, these features are then combined through feature association to yield object information within the visual scene. However, the conventional approach faces limitations when it leverages the template features as a convolution kernel to convolve the search image features, which restricts the ability to capture complex and nonlinear feature transformations of objects, thereby restricting its discriminative capabilities. To overcome this challenge, we propose replacing traditional convolutional correlation with Slot Attention for feature association. This novel approach enables the effective extraction of nonlinear features within the scene, while augmenting the discriminative capacity. Furthermore, to increase the inference efficiency and reduce the parameter occupation, we suggest deploying a single Slot Attention module for multiple associations. Our tracking algorithm, SiamSlot, was evaluated on diverse benchmarks, including VOT2019, GOT-10k, UAV123, and Nfs. The experiments show a remarkable improvement in performance relative to previous methods under the same network size.

UnifiedTT: Visual tracking with unified transformer

Target tracking is an important research task in computer vision. Existing tracking algorithms based on Siamese networks often suffer from the problem of information redundancy between adjacent frames and a lack of ability to capture global dependencies. When similar backgrounds appear around the target, the tracking performance usually significantly decreases. Although target tracking algorithms based on deep convolution and the Transformer have partially addressed these issues, achieving a good balance between the two remains a challenge. In this work, we propose a unified convolution and self-attention Siamese network for target tracking. By utilizing a feature extraction backbone network based on integrated convolution and self-attention styles, we are able to capture globally important regions and key frames while greatly reducing local redundant computations, thereby improving tracking performance. We apply this approach to the task of target tracking and enhance the feature extraction capabilities of both the target template and search template. Experimental results show that our proposed tracking algorithm outperforms some recent classical tracking algorithms, especially achieving improvements of 10.7 % on the high-diversity dataset GOT-10K and 24.7 % on the large-scale and high-quality LaSOT dataset.

Cross-Video Pedestrian Tracking Algorithm with a Coordinate Constraint

Pedestrian tracking in surveillance videos is crucial and challenging for precise personnel management. Due to the limited coverage of a single video, the integration of multiple surveillance videos is necessary in practical applications. In the realm of pedestrian management using multiple surveillance videos, continuous pedestrian tracking is quite important. However, prevailing cross-video pedestrian matching methods mainly rely on the appearance features of pedestrians, resulting in low matching accuracy and poor tracking robustness. To address these shortcomings, this paper presents a cross-video pedestrian tracking algorithm, which introduces spatial information. The proposed algorithm introduces the coordinate features of pedestrians in different videos and a linear weighting strategy focusing on the overlapping view of the tracking process. The experimental results show that, compared to traditional methods, the method in this paper improves the success rate of target pedestrian matching and enhances the robustness of continuous pedestrian tracking. This study provides a viable reference for pedestrian tracking and crowd management in video applications.

Adaptive Kalman Filter for Real-Time Visual Object Tracking Based on Autocovariance Least Square Estimation

Real-time visual object tracking (VOT) may suffer from performance degradation and even divergence owing to inaccurate noise statistics typically engendered by non-stationary video sequences or alterations in the tracked object. This paper presents a novel adaptive Kalman filter (AKF) algorithm, termed AKF-ALS, based on the autocovariance least square estimation (ALS) methodology to improve the accuracy and robustness of VOT. The AKF-ALS algorithm involves object detection via an adaptive thresholding-based background subtraction technique and object tracking through real-time state estimation via the Kalman filter (KF) and noise covariance estimation using the ALS method. The proposed algorithm offers a robust and efficient solution to adapting the system model mismatches or invalid offline calibration, significantly improving the state estimation accuracy in VOT. The computation complexity of the AKF-ALS algorithm is derived and a numerical analysis is conducted to show its real-time efficiency. Experimental validations on tracking the centroid of a moving ball subjected to projectile motion, free-fall bouncing motion, and back-and-forth linear motion, reveal that the AKF-ALS algorithm outperforms a standard KF with fixed noise statistics.