Unmanned Aerial Vehicle Tracking Research Articles

CMDN: Pre-Trained Visual Representations Boost Adversarial Robustness for UAV Tracking

Visual object tracking is widely adopted to unmanned aerial vehicle (UAV)-related applications, which demand reliable tracking precision and real-time performance. However, UAV trackers are highly susceptible to adversarial attacks, while research on developing effective adversarial defense methods for UAV tracking remains limited. To tackle these challenges, we propose CMDN, a novel pre-processing defense network that effectively purifies adversarial perturbations by reconstructing video frames. This network learns robust visual representations from video frames, guided by meaningful features from both the search region and the template. Comprehensive experiments on three benchmarks demonstrate that CMDN is capable of enhancing a UAV tracker’s adversarial robustness in both adaptive and non-adaptive attack scenarios. In addition, CMDN maintains stable defense effectiveness when transferred to heterogeneous trackers. Real-world tests on the UAV platform also validate its reliable defense effectiveness and real-time performance, with CMDN achieving 27 FPS on NVIDIA Jetson Orin 16 GB (25 W mode).

Adaptive Aberrance Repressed Correlation Filters with Cooperative Optimization in High-Dimensional Unmanned Aerial Vehicle Task Allocation and Trajectory Planning

In the rapidly evolving field of unmanned aerial vehicle (UAV) applications, the complexity of task planning and trajectory optimization, particularly in high-dimensional operational environments, is increasingly challenging. This study addresses these challenges by developing the Adaptive Distortion Suppression Correlation Filter Cooperative Optimization (ARCF-ICO) algorithm, designed for high-dimensional UAV task allocation and trajectory planning. The ARCF-ICO algorithm combines advanced correlation filter technologies with multi-objective optimization techniques, enhancing the precision of trajectory planning and efficiency of task allocation. By incorporating weather conditions and other environmental factors, the algorithm ensures robust performance at low altitudes. The ARCF-ICO algorithm improves UAV tracking stability and accuracy by suppressing distortions, facilitating optimal path selection and task execution. Experimental validation using the UAV123@10fps and OTB-100 datasets demonstrates that the ARCF-ICO algorithm outperforms existing methods in Area Under the Curve (AUC) and Precision metrics. Additionally, the algorithm’s consideration of battery consumption and endurance further validates its applicability to current UAV technologies. This research advances UAV mission planning and sets new standards for UAV deployment in both civilian and military applications, where adaptability and accuracy are critical.

Graph-based multi-agent reinforcement learning for collaborative search and tracking of multiple UAVs

This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.

Research on collaborative multi-UAV localization method based on combination navigation information

In challenging environments, unmanned aerial vehicle (UAV) systems often encounter unstable satellite signals and communication link interference. This paper proposes an integrated navigation method that integrates inertial navigation system (INS), global navigation satellite system (GNSS), and visual navigation system (VNS). Utilizing data from onboard sensors, this method merges relative navigation information from feature tracking of multiple UAVs with each UAV’s absolute navigation data. It includes specially designed transmission rules to reduce data exchange between UAVs. Each UAV uses an adaptive unscented Kalman filter (AUKF) method, which is enhanced into a collaborative AUKF (C-AUKF) using a message passing-based approach. Experiments in a simulated mission scenario revealed that the C-AUKF, in comparison to using extended Kalman filter (EKF), significantly improved flight test performance across the entire testing area, with a cumulative deviation of only 10.22 m, about 0.85% of the total flight distance. These results demonstrate that the proposed method not only meets accuracy requirements for position and velocity in integrated navigation but also significantly enhances multi-UAV navigation precision, particularly in scenarios with global positioning system (GPS) interference.

Empowering lightweight detectors: Orientation Distillation via anti-ambiguous spatial transformation for remote sensing images

Knowledge distillation (KD) has been one of the most potential methods to implement a lightweight detector, which plays a significant role in satellite in-orbit processing and unmanned aerial vehicle tracking. However, existing distillation paradigms exhibit limited accuracy in detecting arbitrary-oriented objects represented with rotated bounding boxes in remote sensing images. This issue is attributed to two aspects: (i) boundary discontinuity localization distillation, caused by angle periodicity of rotated bounding boxes, and (ii) spatial ossified feature distillation, induced by orientation-agnostic knowledge transitive regions, both of which contribute to ambiguous orientation estimation of objects. To address these issues, we propose an effective KD method called Orientation Distillation (OD) via anti-ambiguous spatial transformation, which consists of two modules. (i) Anti-ambiguous Location Prediction (ALP) module reformulates the regression transformation between teacher–student bounding boxes as Gaussian distributions fitting procedure. These distributions with distilled potential are optimized to accurately localize objects with the aid of boundary continuity cost. (ii) Orientation-guided Feature Calibration (OFC) module employs a learnable affine matrix to augment fixed CNN sampling grid into a spatially remapped one, which bridges between the multi-scale feature of teacher and student for effectively delivering the refined oriented awareness within adaptively distillation regions. Overall, OD customizes the spatial transformation of bounding box representation and sampling grid to transfer anti-ambiguous orientation knowledge, and significantly improves the performance of lightweight detectors upon non-axially arranged objects. Extensive experiments on multiple datasets demonstrate that our plug-and-play distillation framework achieves state-of-the-art performance. Codes are available at https://github.com/Molly6/OD.

6-DOF UAV Path planning and tracking control for obstacle avoidance: a deep learning-based integrated approach

Path planning is crucial for achieving autonomous and safe flight of fixed-wing unmanned aerial vehicles (UAVs). The existing methods generally cannot simultaneously balance real-time performance, near optimality, and feasibility in three-dimensional obstacle environments. To this end, this paper proposes a deep learning-based integrated path planning and tracking control framework for obstacle avoidance. First, a fundamental framework, including an interfered fluid dynamical system (IFDS)-based path planning module and a backstepping control-based path tracking module, is established. Then, to satisfy high real-time requirements, a long short-term memory (LSTM)-based fast online decision-making network matched with the IFDS is developed. Extensive training samples are generated by the fundamental framework, and receding horizon control is employed to optimize these samples so that the LSTM can provide near-optimal coefficients for the IFDS. Furthermore, the dynamic characteristics of an actual UAV system and the performance of the tracking controller are fully considered during sample generation, enabling that the paths planned by the LSTM-assisted IFDS are highly feasible. Finally, the results of numerical simulations and hardware-in-the-loop experiments show that our method can plan near-optimal obstacle-free paths with high feasibility in real-time.

High-precision tracking and positioning for monitoring Holstein cattle.

Enhanced animal welfare has emerged as a pivotal element in contemporary precision animal husbandry, with bovine monitoring constituting a significant facet of precision agriculture. The evolution of intelligent agriculture in recent years has significantly facilitated the integration of drone flight monitoring tools and innovative systems, leveraging deep learning to interpret bovine behavior. Smart drones, outfitted with monitoring systems, have evolved into viable solutions for wildlife protection and monitoring as well as animal husbandry. Nevertheless, challenges arise under actual and multifaceted ranch conditions, where scale alterations, unpredictable movements, and occlusions invariably influence the accurate tracking of unmanned aerial vehicles (UAVs). To address these challenges, this manuscript proposes a tracking algorithm based on deep learning, adhering to the Joint Detection Tracking (JDT) paradigm established by the CenterTrack algorithm. This algorithm is designed to satisfy the requirements of multi-objective tracking in intricate practical scenarios. In comparison with several preeminent tracking algorithms, the proposed Multi-Object Tracking (MOT) algorithm demonstrates superior performance in Multiple Object Tracking Accuracy (MOTA), Multiple Object Tracking Precision (MOTP), and IDF1. Additionally, it exhibits enhanced efficiency in managing Identity Switches (ID), False Positives (FP), and False Negatives (FN). This algorithm proficiently mitigates the inherent challenges of MOT in complex, livestock-dense scenarios.

Real-time and Intelligent Moving Targets Tracking based on UAV Remote Sensing Video Camera and Brain-like Computing Chips

Abstract. Moving target tracking technology based on Unmanned Aerial Vehicles (UAV) is widely used in many fields such as automatic inspection and emergency response. The existing moving target tracking methods usually have the problems of large computation and low tracking efficiency. Limited by the computing power of the UAV platform, real-time tracking and analysis of multiple targets based on the video data collected by UAV platform is a difficult task. In this paper, we proposed a novel Target Specific Filtering Tracking with Memory (TSFMTrack) method designed for UAV-based real-time tracking tasks, which involves a Tracklet Filtering Module (TFM) for capturing object appearance features and a Tracklet Matching Module (TMM) for bounding box association in each frame. By experimental comparison with other State-Of-The-Art (SOTA) methods on popular MOT and UAV tracking datasets, the TSFMTrack have shown obvious advantages in accuracy, computational efficiency and reliability. Furthermore, we deployed the TSFMTrack on the brain-inspired chip Lynchip KA200, the experimental results have shown that the TSFMTrack is effective on edge computational platform and suitable for UAV real-time tracking tasks.

Vision-Guided Tracking and Emergency Landing for UAVs on Moving Targets

This paper presents a vision-based adaptive tracking and landing method for multirotor Unmanned Aerial Vehicles (UAVs), designed for safe recovery amid propulsion system failures that reduce maneuverability and responsiveness. The method addresses challenges posed by external disturbances such as wind and agile target movements, specifically, by considering maneuverability and control limitations caused by propulsion system failures. Building on our previous research in actuator fault detection and tolerance, our approach employs a modified adaptive pure pursuit guidance technique with an extra adaptation parameter to account for reduced maneuverability, thus ensuring safe tracking of moving objects. Additionally, we present an adaptive landing strategy that adapts to tracking deviations and minimizes off-target landings caused by lateral tracking errors and delayed responses, using a lateral offset-dependent vertical velocity control. Our system employs vision-based tag detection to ascertain the position of the Unmanned Ground Vehicle (UGV) in relation to the UAV. We implemented this system in a mid-mission emergency landing scenario, which includes actuator health monitoring of emergency landings. Extensive testing and simulations demonstrate the effectiveness of our approach, significantly advancing the development of safe tracking and emergency landing methods for UAVs with compromised control authority due to actuator failures.

Anti-UAV410: A Thermal Infrared Benchmark and Customized Scheme for Tracking Drones in the Wild.

The perception of drones, also known as Unmanned Aerial Vehicles (UAVs), particularly in infrared videos, is crucial for effective anti-UAV tasks. However, existing datasets for UAV tracking have limitations in terms of target size and attribute distribution characteristics, which do not fully represent complex realistic scenes. To address this issue, we introduce a generalized infrared UAV tracking benchmark called Anti-UAV410. The benchmark comprises a total of 410 videos with over 438 K manually annotated bounding boxes. To tackle the challenges of UAV tracking in complex environments, we propose a novel method called Siamese drone tracker (SiamDT). SiamDT incorporates a dual-semantic feature extraction mechanism that explicitly models targets in dynamic background clutter, enabling effective tracking of small UAVs. The SiamDT method consists of three key steps: Dual-Semantic RPN Proposals (DS-RPN), Versatile R-CNN (VR-CNN), and Background Distractors Suppression. These steps are responsible for generating candidate proposals, refining prediction scores based on dual-semantic features, and enhancing the discriminative capacity of the trackers against dynamic background clutter, respectively. Extensive experiments conducted on the Anti-UAV410 dataset and three other large-scale benchmarks demonstrate the superior performance of the proposed SiamDT method compared to recent state-of-the-art trackers.

Robust path-following guidance for an autonomous vehicle in the presence of wind

This paper presents a novel nonlinear guidance law for an autonomous unmanned aerial vehicle (UAV) that enables it to track any predefined path while taking into account wind effects. The approach involves treating the desired path as the trajectory of the virtual target, which the UAV tracks without relying on knowledge of the path's curvature. This makes the design applicable to paths with varying levels of complexity and smooth curvatures. Additionally, the proposed method draws inspiration from pursuit guidance, known for its simplicity and ease of implementation. Theoretical analysis ensures that the UAV will converge to the intended path within a fixed time, irrespective of its initial configuration relative to the path. Simulations demonstrate the effectiveness of the proposed method, showcasing the UAV's ability to accurately follow diverse curvilinear paths.

Prescribed time formation synchronous tracking of unmanned aerial vehicles with denial of service attacks

This paper addresses the issue of robust prescribed time synchronous tracking control of unmanned aerial vehicles (UAVs) formation with denial of service attacks. A novel prescribed time synchronous tracking control scheme is proposed within a denial of service attack conversion mechanism. Firstly, on account of the UAV formation in the presence of aerodynamic damping and denial of service attacks, a formation cooperative control model is established under a second-order system. Secondly, to perform formation synchronous tracking within a settling time, a time-varying auxiliary function on account of synchronous state errors is designed for the convergence of prescribed time tracking. The control parameters are designed using a key eigenvalue scaling technique under the effect of denial of service attacks on network topology. The proposed scheme is also extended to UAV formation with a detailed-balanced graph. Finally, simulation results are presented to demonstrate the effectiveness of the proposed strategy.

An omnidirectional spatial monocular visual localization and tracking method for indoor unmanned aerial vehicles based on the two-axis rotary table

Aiming at the complexity and poor adaptability of the calibration process in the traditional unmanned aerial vehicles (UAV) indoor visual positioning, this paper proposes an omnidirectional spatial tracking and localization method for indoor UAV based on the two-axis rotary table. Firstly, the position of the UAV fuselage feature points in the camera coordinate system of the turntable camera is computed by the Pespective-n-Point algorithm utilizing known position information of a plurality of feature points with pixel coordinate information in the corresponding image. Pixel coordinate information is extracted by obtaining UAV body-specific feature points from rotary table camera shots. Then, UAV localization in omnidirectional space can be obtained by using the calibrated rotary axis parameters of the rotary table and the rotation angle of the rotary table and substituting them into Rodriguez’s formula to unify the UAV position information acquired by the rotary table camera at different positions into a unified coordinate system. Finally, the angle at which the rotary table should rotate is calculated from the obtained UAV pose and the spatial position of the camera optical center and the rotary axis of the rotary table. The calculated angle is fed back to the turntable as feedback information. The rotary table receiving the feedback information is rotated to a position where the UAV is located at the center of the camera image. Thereby the tracking and localization of the UAV is realized. The experimental results show that the spatial range of localization is greatly expanded with the localization accuracy reaching the level of binocular visual localization. The omnidirectional spatial tracking and localization of indoor UAV can be conveniently realized by this method.

Saliency-enhanced background-aware correlation filters with dual temporal regularization for unmanned aerial vehicle tracking

Saliency-enhanced background-aware correlation filters with dual temporal regularization for unmanned aerial vehicle tracking

Cross-Parallel Attention and Efficient Match Transformer for Aerial Tracking

Visual object tracking is a key technology that is used in unmanned aerial vehicles (UAVs) to achieve autonomous navigation. In recent years, with the rapid development of deep learning, tracking algorithms based on Siamese neural networks have received widespread attention. However, because of complex and diverse tracking scenarios, as well as limited computational resources, most existing tracking algorithms struggle to ensure real-time stable operation while improving tracking performance. Therefore, studying efficient and fast-tracking frameworks, and enhancing the ability of algorithms to respond to complex scenarios has become crucial. Therefore, this paper proposes a cross-parallel attention and efficient match transformer for aerial tracking (SiamEMT). Firstly, we carefully designed the cross-parallel attention mechanism to encode global feature information and to achieve cross-dimensional interaction and feature correlation aggregation via parallel branches, highlighting feature saliency and reducing global redundancy information, as well as improving the tracking algorithm’s ability to distinguish between targets and backgrounds. Meanwhile, we implemented an efficient match transformer to achieve feature matching. This network utilizes parallel, lightweight, multi-head attention mechanisms to pass template information to the search region features, better matching the global similarity between the template and search regions, and improving the algorithm’s ability to perceive target location and feature information. Experiments on multiple drone public benchmark tests verified the accuracy and robustness of the proposed tracker in drone tracking scenarios. In addition, on the embedded artificial intelligence (AI) platform AGX Xavier, our algorithm achieved real-time tracking speed, indicating that our algorithm can be effectively applied to UAV tracking scenarios.

UAV Target Tracking Algorithm Based on Illumination Adaptation and Future Awareness in Low Illumination Scenes

Aiming at problems such as tracking failure caused by illumination changes often encountered during unmanned aerial vehicle (UAV) tracking, a target tracking algorithm with illumination adaptive and future-aware correlation filters is proposed based on the background-aware correlation filters (BACF) algorithm, which realizes reliable UAV tracking tasks at night. First, the dark scene is recognized, and an efficient image enhancement module is used to enhance the brightness of low illumination images. Then a future-aware module is constructed to train the tracking model using the contextual information of the target in the next frame for better robustness. Finally, the model updating stage involves adaptive filter updating and adaptive learning rate updating to enhance target tracking precision. The results of the comparison experiments with the state-of-the-art algorithms on the UAVDark135, UAVDT, and DTB70 datasets show that the algorithm in this paper outperforms the state-of-the-art tracking methods and has better tracking performance under light changes and fast motion (FM) scenarios. The tracking speed on a single Central processing unit (CPU) reaches 49 FPS, which satisfies the real-time requirement of UAV tracking.

Real-Time Optical Localization and Tracking of UAV Using Ellipse Detection

A real-time three-dimensional (3D) spatial localization system to track a UAV (unmanned aerial vehicle) using a single off-board conventional camera and an onboard circle-shaped colored Light-Emitting Diode (LED) marker is presented. The image from the camera is color-segmented and then morphologically closed before being sent to the ellipse detector. The location of the drone is then estimated with geometric/optical calculations using the pixel coordinates provided by the real-time ellipse detector. The location data generated by this system was validated by a Motion-Capture System (MCS) that was simultaneously tracking the system in real-time. The average position error was of 6cm, while the processing speed achieved 15 locations/second, which are comparable with recent research references. The usage of a low-cost camera makes this method promising for most mobile UAV tracking applications.

Self-Prompting Tracking: A Fast and Efficient Tracking Pipeline for UAV Videos

In the realm of visual tracking, remote sensing videos captured by Unmanned Aerial Vehicles (UAVs) have seen significant advancements with wide applications. However, there remain challenges to conventional Transformer-based trackers in balancing tracking accuracy and inference speed. This problem is further exacerbated when Transformers are extensively implemented at larger model scales. To address this challenge, we present a fast and efficient UAV tracking framework, denoted as SiamPT, aiming to reduce the number of Transformer layers without losing the discriminative ability of the model. To realize it, we transfer the conventional prompting theories in multi-model tracking into UAV tracking, where a novel self-prompting method is proposed by utilizing the target’s inherent characteristics in the search branch to discriminate targets from the background. Specifically, a self-distribution strategy is introduced to capture feature-level relationships, which segment tokens into distinct smaller patches. Subsequently, salient tokens within the full attention map are identified as foreground targets, enabling the fusion of local region information. These fused tokens serve as prompters to enhance the identification of distractors, thereby avoiding the demand for model expansion. SiamPT has demonstrated impressive results on the UAV123 benchmark, achieving success and precision rates of 0.694 and 0.890 respectively, while maintaining an inference speed of 91.0 FPS.

Achieving stable trajectory tracking in complex environments using an adaptive PID control strategy-based quadcopter drone

Stable trajectory tracking of unmanned aerial vehicles (UAVs) in complex environments is of paramount importance for achieving high precision and robustness in flight missions. In this paper, we address the attitude control problem of quadrotor UAVs and propose an optimization method based on adaptive PID control strategy. This text first presents an overview of the current status of UAVs both domestically and internationally, followed by the establishment of a mathematical model for quadrotor UAVs. Next, analysing the application of the traditional PID algorithm in UAV attitude control and provide a detailed description of the principles behind genetic algorithms and simulated annealing algorithms, along with their application in optimizing PID parameters. Through simulation experiments conducted in strong wind conditions, we compare the performance of the traditional PID algorithm with the optimization algorithm in stable trajectory tracking tasks. The experimental results demonstrate that the optimization algorithm significantly enhances the flight stability and accuracy of UAVs. Finally, this text summarizes the research findings and provide an outlook on future development directions.

AI-Embedded UAV System for Detecting and Pursuing Unwanted UAVs

In recent years, the use of unmanned aerial vehicle (UAV) platforms in civil and military applications has surged, highlighting the critical role of artificial intelligence (AI) embedded UAV systems in the future. This study introduces the Autonomous Drone (Vechür-SIHA), a novel AI-embedded UAV system designed for real-time detection and tracking of other UAVs during flight sequences. Leveraging advanced object detection algorithms and an LSTM-based tracking mechanism, our system achieves an impressive 80% accuracy in drone detection, even in challenging conditions like varying backgrounds and adverse weather.
 Our system boasts the capability to simultaneously track multiple drones within its field of view, maintaining flight for up to 35 minutes, making it ideal for extended missions that require continuous UAV tracking. Moreover, it can lock onto and track other UAVs in mid-air for durations of 4-10 seconds without losing contact, a feature with significant potential for security applications.
 This research marks a substantial contribution to the development of AI-embedded UAV systems, with broad implications across diverse domains such as search and rescue operations, border security, and forest fire prevention. These results provide a solid foundation for future research, fostering the creation of similar systems tailored to different applications, ultimately enhancing the efficiency and safety of UAV operations. The novel approach to real-time UAV detection and tracking presented here holds promise for driving innovations in UAV technology and its diverse applications.