Visual Tracking Method Research Articles

Transformer object tracking algorithm based on spatiotemporal template focal attention

Existing visual object tracking methods only use the target area of the first frame as a template, which makes it easy to fail in fast-changing and complex backgrounds. To solve this problem, this paper proposes a Transformer-based object tracking algorithm that focuses on the target focus information in the template and dynamically updates the template features. In order to reduce the interference of background information on attention, this algorithm uses a sparse Transformer module to achieve the interaction of feature information; a template focus attention module is also proposed to focus on the dynamic template features and the initial template features to retain the highly reliable feature information in the initial template. Experimental results show that the success rate and accuracy of this algorithm in the 0TB100 benchmark test are and respectively 70.9%, 91.6%and the success rate and accuracy are improved by 4.11%and respectively compared with the similar template update algorithm STARK 3.27%. This algorithm effectively addresses the limitations of existing visual object tracking methods and improves the accuracy and robustness of tracking.

Optimal visual control of tendon-sheath-driven continuum robots with robust Jacobian estimation in confined environments

Accurate control of continuum robots in confined environments presents a significant challenge due to the need for a precise kinematic model, which is susceptible to external interference. This paper introduces a model-less optimal visual control (MLOVC) method that enables a tendon-sheath-driven continuum robot (TSDCR) to effectively track visual targets in a confined environment while ensuring stability. The method allows for intraluminal navigation of TSDCRs along narrow lumens. To account for the presence of external outliers, a robust Jacobian estimation method is proposed, wherein improved iterative reweighted least squares with sliding windows are used to online calculate the robot’s Jacobian matrix from sensing data. The estimated Jacobian establishes the motion relationship between the visual feature and the actuation. Furthermore, an optimal visual control method based on quadratic programming (QP) is designed for visual target tracking, while considering the robot’s physical constraint and control constraints. The MLOVC method for visual tracking provides a reliable alternative that does not rely on the precise kinematics of TSDCRs and takes into consideration the impact of outliers. The control stability of the proposed approach is demonstrated through Lyapunov analysis. Simulations and experiments are conducted to evaluate the effectiveness of the MLOVC method, and the results demonstrate that it enhances tracking performance in terms of accuracy and stability.

Active Learning for Deep Visual Tracking.

Convolutional neural networks (CNNs) have been successfully applied to the single target tracking task in recent years. Generally, training a deep CNN model requires numerous labeled training samples, and the number and quality of these samples directly affect the representational capability of the trained model. However, this approach is restrictive in practice, because manually labeling such a large number of training samples is time-consuming and prohibitively expensive. In this article, we propose an active learning method for deep visual tracking, which selects and annotates the unlabeled samples to train the deep CNN model. Under the guidance of active learning, the tracker based on the trained deep CNN model can achieve competitive tracking performance while reducing the labeling cost. More specifically, to ensure the diversity of selected samples, we propose an active learning method based on multiframe collaboration to select those training samples that should be and need to be annotated. Meanwhile, considering the representativeness of these selected samples, we adopt a nearest-neighbor discrimination method based on the average nearest-neighbor distance to screen isolated samples and low-quality samples. Therefore, the training samples' subset selected based on our method requires only a given budget to maintain the diversity and representativeness of the entire sample set. Furthermore, we adopt a Tversky loss to improve the bounding box estimation of our tracker, which can ensure that the tracker achieves more accurate target states. Extensive experimental results confirm that our active-learning-based tracker (ALT) achieves competitive tracking accuracy and speed compared with state-of-the-art trackers on the seven most challenging evaluation benchmarks. Project website: https://sites.google.com/view/altrack/.

A Target Sensing and Visual Tracking Method for Countering Unmanned Aerial Vehicle Swarm

A Target Sensing and Visual Tracking Method for Countering Unmanned Aerial Vehicle Swarm

Walking representation and simulation based on multi-source image fusion and multi-agent reinforcement learning for gait rehabilitation

In the formulation of strategies for walking rehabilitation, achieving precise identification of the current state and making rational predictions about the future state are crucial but often unrealized. To tackle this challenge, our study introduces a unified framework that integrates a novel 3D walking motion capture method using multi-source image fusion and a walking rehabilitation simulation approach based on multi-agent reinforcement learning. We found that, (i) the proposal achieved an accurate 3D walking motion capture and outperforms other advanced methods. Experimental evidence indicates that, compared to similar visual skeleton tracking methods, the proposed approach yields results with higher Pearson correlation (r=0.93), intra-class correlation coefficient (ICC(2,1)=0.91), and narrower confidence intervals ([0.90,0.95] for r, [0.88,0.94] for ICC(2,1)) when compared to standard results. The outcomes of the proposed approach also exhibit commendable correlation and concurrence with those obtained through the IMU-based skeleton tracking method in the assessment of gait parameters ([0.85,0.89] for r, [0.75,0.81] for ICC(2,1)); (ii) multi-agent reinforcement learning has the potential to be used to solve the simulation task of gait rehabilitation. In mimicry experiment, our proposed simulation method for gait rehabilitation not only enables the intelligent agent to converge from the initial state to the target state, but also observes evolutionary patterns similar to those observed in clinical practice through motor state resolution. This study offers valuable contributions to walking rehabilitation, enabling precise assessment and simulation-based interventions, with potential implications for clinical practice and patient outcomes.

Experimental study of the wall particle motion in a screw reactor

Screw reactors are gas/solid systems widely employed in industrial applications. Understanding the local powder mixing within these reactors is crucial, particularly at the reactor wall where wall/solid heat transfer occurs. This study proposes a non-invasive measurement method utilizing optical devices to assess wall particle motion. Two techniques for evaluating the particle motion are compared: a simple visual tracking method and particle image velocimetry. Both methods yield similar results, demonstrating consistent wall particle motion across the tube periphery during experimentation. The influence of operational and geometric parameters has been investigated, revealing that filling degree, powder flowability, and screw geometry significantly impact wall particle motion. Symbolic regression has been employed to derive dimensionless models that effectively predict experimental data. These models can be utilized to forecast wall particle motion, serving as a valuable tool for screw conveyor design.

Visual Sensor Target Tracking and Localization Method for Automatic Excavators

Visual Sensor Target Tracking and Localization Method for Automatic Excavators

Visual tracking based on spatiotemporal transformer and fusion sequences

Currently, Transformer-based visual tracking methods have exhibited impressive performance. However, despite their widespread adoption, they still have certain limitations. For example, the design of the Transformer framework is somewhat original and redundant, resulting in lower efficiency. In addition, their application methods lack time consideration, and there is a lack of spatiotemporal correlation between tracking video sequences and predicting coordinate sequences, making it difficult to effectively integrate, and the robustness of corresponding tracking templates is insufficient. To address these issues, we propose a new visual tracking method (STFS). Firstly, it introduces a novel Flatten Transformer architecture, which, in comparison to previous modules, offers enhanced efficiency and expressiveness. Secondly, it takes multi frame feature maps and bounding box coordinates as inputs, integrates spatiotemporal information through the spatiotemporal sequence attention module, and provides relevant sequences for historical trend prediction. Finally, it uses diffusion methods to construct tracking templates and improve stability. To verify the performance of the tracker, we conducted experiments on benchmark datasets including GOT-10 K, LaSOT, TrackingNet, VOT2020, OTB100, and UAV123. The results demonstrate that STFS has achieved competitive experimental results.

Learning Dynamic Compact Memory Embedding for Deformable Visual Object Tracking.

Recently, template-based trackers have become the leading tracking algorithms with promising performance in terms of efficiency and accuracy. However, the correlation operation between query feature and the given template only achieves accurate target localization, but is prone to state estimation error, especially when the target suffers from severe deformation. To address this issue, segmentation-based trackers are proposed that use per-pixel matching to improve the tracking performance of deformable objects effectively. However, most of the existing trackers only match with the target features of the initial frame, thereby lacking the discrimination for handling a variety of challenging factors, e.g., similar distractors, background clutter, and appearance change. To this end, we propose a dynamic compact memory embedding technique to enhance the discrimination of the segmentation-based visual tracking method that can well tell the target from the background. Specifically, we initialize a memory embedding with the target features in the first frame. During the tracking process, the current target features that have certain correlation with the existing memory are updated to the memory embedding online. To further improve the tracking accuracy for deformable objects, we use a weighted point-to-global matching strategy to measure the correlation between the pixelwise query feature and the whole template, so as to capture more detailed deformation information. Extensive evaluations on six challenging tracking benchmarks including VOT2016, VOT2018, VOT2019, GOT-10K, TrackingNet, and LaSOT demonstrate the superiority of our method over recent remarkable trackers. Besides, our tracker outperforms the excellent segmentation-based trackers, i.e., D3S and SiamMask on the DAVIS2017 benchmark. The code is available at https://github.com/peace-love243/CMEDFL.

Human-Following Strategy for Orchard Mobile Robot Based on the KCF-YOLO Algorithm

Autonomous mobile robots play a vital role in the mechanized production of orchards, where human-following is a crucial collaborative function. In unstructured orchard environments, obstacles often obscure the path, and personnel may overlap, leading to significant disruptions to human-following. This paper introduces the KCF-YOLO fusion visual tracking method to ensure stable tracking in interference environments. The YOLO algorithm provides the main framework, and the KCF algorithm intervenes in assistant tracking. A three-dimensional binocular-vision reconstruction method was used to acquire personnel positions, achieving stabilized visual tracking in disturbed environments. The robot was guided by fitting the personnel’s trajectory using an unscented Kalman filter algorithm. The experimental results show that, with 30 trials in multi-person scenarios, the average tracking success rate is 96.66%, with an average frame rate of 8 FPS. Additionally, the mobile robot is capable of maintaining a stable following speed with the target individuals. Across three human-following experiments, the horizontal offset Error Y does not exceed 1.03 m. The proposed KCF-YOLO tracking method significantly bolsters the stability and robustness of the mobile robot for human-following in intricate orchard scenarios, offering an effective solution for tracking tasks.

An omnidirectional spatial monocular visual localization and tracking method for indoor unmanned aerial vehicles based on the two-axis rotary table

Aiming at the complexity and poor adaptability of the calibration process in the traditional unmanned aerial vehicles (UAV) indoor visual positioning, this paper proposes an omnidirectional spatial tracking and localization method for indoor UAV based on the two-axis rotary table. Firstly, the position of the UAV fuselage feature points in the camera coordinate system of the turntable camera is computed by the Pespective-n-Point algorithm utilizing known position information of a plurality of feature points with pixel coordinate information in the corresponding image. Pixel coordinate information is extracted by obtaining UAV body-specific feature points from rotary table camera shots. Then, UAV localization in omnidirectional space can be obtained by using the calibrated rotary axis parameters of the rotary table and the rotation angle of the rotary table and substituting them into Rodriguez’s formula to unify the UAV position information acquired by the rotary table camera at different positions into a unified coordinate system. Finally, the angle at which the rotary table should rotate is calculated from the obtained UAV pose and the spatial position of the camera optical center and the rotary axis of the rotary table. The calculated angle is fed back to the turntable as feedback information. The rotary table receiving the feedback information is rotated to a position where the UAV is located at the center of the camera image. Thereby the tracking and localization of the UAV is realized. The experimental results show that the spatial range of localization is greatly expanded with the localization accuracy reaching the level of binocular visual localization. The omnidirectional spatial tracking and localization of indoor UAV can be conveniently realized by this method.

Surgical Significance of Variant Anatomy of Inferior Mesenteric Artery and Left Colic Artery: A Computed Tomography Angiographic Study

Introduction: A variation in vessel morphology determines flow dynamics and vascular disease pathogenesis. Definite information on the distinctive anatomical variations of the mesenteric vessels is extremely important for laparoscopic surgical procedures and interventional radiological procedures in the abdominal region. Three-dimensional-computed tomography angiography (3D-CTA) is a less invasive method to evaluate vascular anatomy using a visual tracking method on high-definition images, contrast-enhanced inferior mesenteric artery (IMA) can be traced to its terminal branches beside the colon wall. Subjects and Methods: Abdominal contrast-enhanced computed tomography scan data from 180 patients was retrospectively collected, reconstructed, and analyzed. The origin, length of IMA from origin to the first branch, branching patterns of IMA, and tracking patterns of the ascending branch of left colic artery (LCA), were examined, and their associations with clinical features were analyzed. Results: IMA displayed variations in the level of origin ranging from upper 1/3 of L3 to upper 1/3 of L4, mean length from origin to the first branch was 27.4 ± 7.8 mm. Observed branching patterns of IMA were grouped according to Yada classification: Type 1 (52.2%), Type 2 (15.5%), Type 3 (14.4%), and Type 4 (17.7%). Tracking of the ascending branch of LCA before anastomosis with marginal artery revealed four patterns: Type A LCA pattern in 43.9%; Type B LCA anatomy pattern was (19.4%); Type C LCA anatomy pattern was (20.0%); and Type D LCA anatomy pattern was observed (16.7%). Conclusion: Preoperative understanding of the vascular variations of IMA and LCA can be obtained by 3D-CTA. This may be helpful to surgeons in planning preoperative strategies to prevent iatrogenic injuries.

Learning Cross-Attention Discriminators via Alternating Time-Space Transformers for Visual Tracking.

In the past few years, visual tracking methods with convolution neural networks (CNNs) have gained great popularity and success. However, the convolution operation of CNNs struggles to relate spatially distant information, which limits the discriminative power of trackers. Very recently, several Transformer-assisted tracking approaches have emerged to alleviate the above issue by combining CNNs with Transformers to enhance the feature representation. In contrast to the methods mentioned above, this article explores a pure Transformer-based model with a novel semi-Siamese architecture. Both the time-space self-attention module used to construct the feature extraction backbone and the cross-attention discriminator used to estimate the response map solely leverage attention without convolution. Inspired by the recent vision transformers (ViTs), we propose the multistage alternating time-space Transformers (ATSTs) to learn robust feature representation. Specifically, temporal and spatial tokens at each stage are alternately extracted and encoded by separate Transformers. Subsequently, a cross-attention discriminator is proposed to directly generate response maps of the search region without additional prediction heads or correlation filters. Experimental results show that our ATST-based model attains favorable results against state-of-the-art convolutional trackers. Moreover, it shows comparable performance with recent "CNN + Transformer" trackers on various benchmarks while our ATST requires significantly less training data.

Prediction during spoken language processing in monolingual and multilingual children

Abstract Listeners use linguistic cues to anticipate upcoming words, but not all do so to the same extent. While we know that monolingual children use prediction during spoken language processing from a young age and that adult L2 speakers may sometimes be slower, very few studies have focused on bilingual or multilingual children. Moreover, previous research suggests that literacy boosts anticipation in spoken language processing, but this has not been tested yet in bi/multilinguals. We investigated linguistic prediction and its relation to reading and vocabulary skills in 38 eight- to twelve-year-old bilingual and multilingual children who speak different heritage languages and Italian as the majority language, in comparison to 32 age-matched monolingual Italian children. Using a visual world eye tracking method, we tested children’s ability to anticipate nouns based on morphosyntactic cues (gender- and number-marked articles) in Italian. The results show efficient prediction in both groups, although monolinguals were faster than bi/multilinguals. While we found a positive relation between predictive language processing and reading in monolingual children, there were no reliable effects in bilingual and multilingual children. Future work is required to better understand the relation between prediction and literacy in this population.

Spatio-temporal interactive fusion based visual object tracking method

Visual object tracking tasks often struggle with utilizing inter-frame correlation information and handling challenges like local occlusion, deformations, and background interference. To address these issues, this paper proposes a spatio-temporal interactive fusion (STIF) based visual object tracking method. The goal is to fully utilize spatio-temporal background information, enhance feature representation for object recognition, improve tracking accuracy, adapt to object changes, and reduce model drift. The proposed method incorporates feature-enhanced networks in both temporal and spatial dimensions. It leverages spatio-temporal background information to extract salient features that contribute to improved object recognition and tracking accuracy. Additionally, the model’s adaptability to object changes is enhanced, and model drift is minimized. A spatio-temporal interactive fusion network is employed to learn a similarity metric between the memory frame and the query frame by utilizing feature enhancement. This fusion network effectively filters out stronger feature representations through the interactive fusion of information. The proposed tracking method is evaluated on four challenging public datasets. The results demonstrate that the method achieves state-of-the-art (SOTA) performance and significantly improves tracking accuracy in complex scenarios affected by local occlusion, deformations, and background interference. Finally, the method achieves a remarkable success rate of 78.8% on TrackingNet, a large-scale tracking dataset.

Visual Tracking based on deformable Transformer and spatiotemporal information

At present, the Transformer-based Siamese network visual tracking method has shown strong influence and achieved remarkable results on various experimental sets. Especially on the premise of training on large-scale datasets, attention-based Transformer structures have been widely used. However, many trackers ignore the fusion enhancement of local and global features, and lack the extraction of spatiotemporal information. At the same time, the original Transformer structural features are redundant and will be affected by irrelevant parts beyond the region of interest. To address these issues, we propose a new method (DTS) based on deformable Transformer and spatiotemporal information. As a Siamese structure, it contains multiple modules. The template frame gets local to global important features through 2D CNN and Self-Attention. The search frame gets spatiotemporal information of interest through 3D CNN and SFM and DAM and then uses Cross-Attention to establish the correlation between them, and finally predict the bounding box of the target through the corner points. In order to verify the effectiveness of our method, we conduct experiments on the LaSOT, GOT-10K, TrackingNet, VOT2018, OTB100 and VAU123 benchmark datasets, the result index is 2%–3% higher than the baseline method. The model structure is simplified without affecting the performance, and the FPS reaches 50. The results show that our proposed tracker is very competitive compared with other state-of-the-art methods.

The New Kernel-based Multiple Instances Learning Algorithm for Object Tracking

Background:: Visual tracking is a crucial component of computer vision systems. Objective:: To deal with the problems of occlusion, pose variation, and illumination in long-time tracking, we propose a new kernel-based multiple instances learning tracker. objective: visual tracking Method:: The tracker captures five positive bags, including the occlusion bag, pose bag, illumination bag, scale bag, and object bag, to deal with the appearance changes of an object in a complex environment. A Gaussian kernel function is used to compute the inner product for selecting the powerful weak classifiers, which further improves the efficiency of the tracker. Moreover, the tracking situation is determined by using these five classifiers, and the correlating classifiers are updated. Results:: The experimental results show that the proposed algorithm is robust in terms of occlusion and various appearance changes. Conclusion:: The proposed algorithm preforms well in complex situations. conclusion: And it preforms well in complex situation.

Discriminative learning of online appearance modeling methods for visual tracking

Discriminative learning of online appearance modeling methods for visual tracking

An Internet of Electronic-Visual Things Indoor Localization System Using Adaptive Kalman Filter

In recent years, various indoor social security emergencies have brought major changes to people’s lives. However, the state-of-art surveillance technology achieving object tracking alone is insufficient through a single visual tracking method and may cause inaccurate tracking (such as burred faces, loss target, and occlusion) and is time-consuming for a long video sequence. Accurate tracking and localization systems are essential for developing indoor localization systems. This paper designs a fusing communication system called the Internet of Electronic-Visual Things Localization system (IoEVT), which simultaneously tracks targets assisting with an Adaptive Kalman Filter (AKF). The data fusion indoor localization approach using AKF fuses multi-data from electronic (E) and visual (V) data, such as WiFi and cameras. The fusing method significantly improves EV data sensing abilities for solving occluded visual information when only using single V data in occlusion scenarios and reduces the localization error rate. In addition, our real-world experimental validation shows that the proposed IoEVT system achieves better accuracy, scalability, and robustness with low network bandwidth consumption compared with the single E or V data in many scenarios, such as occlusion and occlusion-free.

Comprehensive study on visual tracking methods and its application on aerial videos

Comprehensive study on visual tracking methods and its application on aerial videos