Unmanned Aerial Vehicle Target Tracking Research Articles

A Dynamic Visual SLAM System Incorporating Object Tracking for UAVs

The capability of unmanned aerial vehicles (UAVs) to capture and utilize dynamic object information assumes critical significance for decision making and scene understanding. This paper presents a method for UAV relative positioning and target tracking based on a visual simultaneousocalization and mapping (SLAM) framework. By integrating an object detection neural network into the SLAM framework, this method can detect moving objects and effectively reconstruct the 3D map of the environment from image sequences. For multiple object tracking tasks, we combine the region matching of semantic detection boxes and the point matching of the optical flow method to perform dynamic object association. This joint association strategy can prevent trackingoss due to the small proportion of the object in the whole image sequence. To address the problem ofacking scale information in the visual SLAM system, we recover the altitude data based on a RANSAC-based plane estimation approach. The proposed method is tested on both the self-created UAV dataset and the KITTI dataset to evaluate its performance. The results demonstrate the robustness and effectiveness of the solution in facilitating UAV flights.

Unmanned Aerial Vehicle Fire Detection Platform Based on Semantic Yolov5 and Autonomous Recognition

In recent years, as fire image detection has become a research hotspot. One class of methods is color-based methods, which are very sensitive to brightness and shadows. As a result, the number of false alarms generated by these methods is high. Aiming at the task requirements of airborne binocular vision obstacle avoidance and target tracking, this paper establishes the verification platform architecture of UAV (Unmanned Aerial Vehicle) binocular vision obstacle avoidance and target tracking. For the update and maintenance of boundary regions, we can also continuously extract richer information from the boundary, make more elaborate plans, and develop an incremental method to detect locally updated maps within the boundary. The fire point can be independently and quickly identified through deep learning to extinguish the fire accurately. Assuming that the system incorrectly identifies 2 out of 80 non-fire sources as fire sources, so the results indicate a precision of about 88%, a recall of 90%. However, the traditional fire detection is around 80%.

Vision-based finite-time prescribed performance control for uncooperative UAV target-tracking subject to field of view constraints

This paper presents a vision-based finite-time prescribed performance controller for unmanned aerial vehicle (UAV) tracking of uncooperative aerial targets. The relative states between UAV and target are estimated by an onboard monocular camera. The inability of visual measurements to accurately determine the initial state of the target renders conventional prescribed performance controllers ineffective in such situations. As a result, it becomes essential to address the problem of prescribed performance control under conditions of uncertain initial values By utilizing an auxiliary transforming function, an Asymmetric Barrier Lyapunov Function (ABLF) and a finite-time prescribed performance function, a robust adaptive controller based on backstepping framework is proposed to deal with state constraints under unknown initial tracking conditions. It is proved that, the closed-loop relative position is capable of reaching the prescribed performance bound before the preset transforming time and converging to the prescribed steady-state error before a finite setting time. Simulation examples are provided to illustrated the effectiveness of the proposed tracking algorithm.

Technological Innovation: New Unmanned Aerial Vehicle Target Tracking Technology

This article mainly discusses a new type of unmanned aerial vehicle target tracking technology, comprising a drone using a built-in GPS module and a device with a built-in GPS module worn by a human body. When the human body is in motion, the device worn by the human body and the drone generate pseudocodes; Perform pseudorange measurement at a certain moment in GPS time; Obtain a nonlinear equation system related to pseudorange, repeat iterative calculations to obtain accurate coordinates, use Dijkstra algorithm to calculate the shortest path in a weighted directed graph, and the drone flies according to the shortest movement trajectory from the starting point to the human target point; The drone and device of the present invention perform pseudorange measurement at a certain moment in GPS time, and repeat iterative calculations to reduce errors, which is conducive to obtaining accurate coordinates of the device and drone; Based on weighted graph search, the coordinates of the drone and the human body are treated as two endpoints, representing the starting and ending points. The algorithm calculates the shortest trajectory in the graph.

UAV Target Tracking Algorithm Based on Illumination Adaptation and Future Awareness in Low Illumination Scenes

Aiming at problems such as tracking failure caused by illumination changes often encountered during unmanned aerial vehicle (UAV) tracking, a target tracking algorithm with illumination adaptive and future-aware correlation filters is proposed based on the background-aware correlation filters (BACF) algorithm, which realizes reliable UAV tracking tasks at night. First, the dark scene is recognized, and an efficient image enhancement module is used to enhance the brightness of low illumination images. Then a future-aware module is constructed to train the tracking model using the contextual information of the target in the next frame for better robustness. Finally, the model updating stage involves adaptive filter updating and adaptive learning rate updating to enhance target tracking precision. The results of the comparison experiments with the state-of-the-art algorithms on the UAVDark135, UAVDT, and DTB70 datasets show that the algorithm in this paper outperforms the state-of-the-art tracking methods and has better tracking performance under light changes and fast motion (FM) scenarios. The tracking speed on a single Central processing unit (CPU) reaches 49 FPS, which satisfies the real-time requirement of UAV tracking.

Infrared Small UAV Target Detection Algorithm Based on Enhanced Adaptive Feature Pyramid Networks

Under the explosive growth of “low-slow-small” unmanned aerial vehicles (UAVs), harassment incidents with “low-slow-small” UAVs, such as illegal surveying and mapping, no-fly zone flights, terrorist attacks, have frequently been occurring due to the low entry barrier to the industry and the lack of market norms and control standards, and severely impair the safety of public life and property. The premise of developing anti-UAV technology is efficient and accurate UAV target detection, which plays a vital role in the follow-up UAV target tracking, attack, and interception. This paper proposes a new DEAX algorithm named enhanced adaptive feature pyramid networks-based Target Detection Algorithm for infrared small UAV target detection. Our proposed algorithm improves the original feature pyramid networks in four aspects for the small UAV target detection task. 1) Use channel separation to keep more channels when adjusting the number of channels in convolutional to avoid the loss of helpful feature information; 2) Design a feature enhancement module to enhance “target feature” and suppress “non-target feature”; 3) Alleviate the differences of receptive fields and semantic information among different layers by shared convolution; 4) Introduce adaptive feature fusion method into feature fusion and complete the construction of enhanced adaptive feature pyramid networks (EAFPN) to solve the problem of weakening feature expression in cross-scale feature fusion. EAFPN is applied to the single-stage target detection networks. We find the detection accuracy and speed of the algorithm outperform those based on feature pyramid networks (FPN), where the improvement on mAP is 8.9%.

Long-Term Target Tracking of UAVs Based on Kernelized Correlation Filter

During the target tracking process of unmanned aerial vehicles (UAVs), the target may disappear from view or be fully occluded by other objects, resulting in tracking failure. Therefore, determining how to identify tracking failure and re-detect the target is the key to the long-term target tracking of UAVs. Kernelized correlation filter (KCF) has been very popular for its satisfactory speed and accuracy since it was proposed. It is very suitable for UAV target tracking systems with high real-time requirements. However, it cannot detect tracking failure, so it is not suitable for long-term target tracking. Based on the above research, we propose an improved KCF to match long-term target tracking requirements. Firstly, we introduce a confidence mechanism to evaluate the target tracking results to determine the status of target tracking. Secondly, the tracking model update strategy is designed to make the model suffer from less background information interference, thereby improving the robustness of the algorithm. Finally, the Normalized Cross Correlation (NCC) template matching is used to make a regional proposal first, and then the tracking model is used for target re-detection. Then, we successfully apply the algorithm to the UAV system. The system uses binocular cameras to estimate the target position accurately, and we design a control method to keep the target in the UAV’s field of view. Our algorithm has achieved the best results in both short-term and long-term evaluations of experiments on tracking benchmarks, which proves that the algorithm is superior to the baseline algorithm and has quite good performance. Outdoor experiments show that the developed UAV system can achieve long-term, autonomous target tracking.

Multitarget Real-Time Tracking Algorithm for UAV IoT

Unmanned aerial vehicles (UAVs) have increased the convenience of urban life. Representing the recent rapid development of drone technology, UAVs have been widely used in fifth-generation (5G) cellular networks and the Internet of Things (IoT), such as drone aerial photography, express drone delivery, and drone traffic supervision. However, owing to low altitude and low speed, drones can only limitedly monitor and detect small target objects, resulting in frequent intrusion and collision. Traditional methods of monitoring the safety of drones are mostly expensive and difficult to implement. In smart city construction, a large number of smart IoT cameras connected to 5G networks are installed in the city. Captured drone images are transmitted to the cloud via a high-speed and low-latency 5G network, and machine learning algorithms are used for target detection and tracking. In this study, we propose a method for real-time tracking of drone targets by using the existing monitoring network to obtain drone images in real time and employing deep learning methods by which drones in urban environments can be guided. To achieve real-time tracking of UAV targets, we employed the tracking-by-detection mode in machine learning, with the network-modified YOLOv3 (you only look once v3) as the target detector and Deep SORT as the target tracking correlation algorithm. We established a drone tracking dataset that contains four types of drones and 2800 pictures in different environments. The tracking model we trained achieved 94.4% tracking accuracy in real-time UAV target tracking and a tracking speed of 54 FPS. These results comprehensively demonstrate that our tracking model achieves high-precision real-time UAV target tracking at a reduced cost.

MultiRPN-DIDNet: Multiple RPNs and Distance-IoU Discriminative Network for Real-Time UAV Target Tracking

Target tracking in low-altitude Unmanned Aerial Vehicle (UAV) videos faces many technical challenges due to the relatively small sizes, various orientation changes of the objects and diverse scenes. As a result, the tracking performance is still not satisfactory. In this paper, we propose a real-time single-target tracking method with multiple Region Proposal Networks (RPNs) and Distance-Intersection-over-Union (Distance-IoU) Discriminative Network (DIDNet), namely MultiRPN-DIDNet, in which ResNet50 is used as the backbone network for feature extraction. Firstly, an instance-based RPN suitable for the target tracking task is constructed under the framework of Simases Neural Network. RPN is to perform bounding box regression and classification, in which channel attention mechanism is integrated to improve the representative capability of the deep features. The RPNs built on the Block 2, Block 3 and Block 4 of ResNet50 output their own Regression (Reg) coefficients and Classification scores (Cls) respectively, which are weighted and then fused to determine the high-quality region proposals. Secondly, a DIDNet is designed to correct the candidate target’s bounding box finely through the fusion of multi-layer features, which is trained with the Distance-IoU loss. Experimental results on the public datasets of UAV20L and DTB70 show that, compared with the state-of-the-art UAV trackers, the proposed MultiRPN-DIDNet can obtain better tracking performance with fewer region proposals and correction iterations. As a result, the tracking speed has reached 33.9 frames per second (FPS), which can meet the requirements of real-time tracking tasks.

BP neural Network-Kalman filter fusion method for unmanned aerial vehicle target tracking

The poor real-time performance and target occlusion occurred easily when the UAV was tracking the target. In this paper, a target tracking method based on the Back Propagation neural network fusion Kalman filter algorithm was developed to solve the position prediction problem of the UAV target tracking in real time. Firstly, the target tracking algorithm was used to acquire the center position coordinates of the target on the onboard computer, and then the coordinate difference matrix was constructed to train the BP neural network in real time. Secondly, when the target was occluded by the obstacles judged by the Bhattacharyya coefficient, the BP neural network fusion Kalman filter algorithm was used to accurately predict the center position coordinates of the occluded target. Then the flight speed of UAV was calculated by the deviation between the coordinates of the target and the image center. Finally, the velocity command was sent to the UAV by the onboard computer. The experimental results shown that the target position predicted by BP neural network fusion Kalman filter algorithm was more accurate and robust in predicting the center position coordinates of the target, and the UAV can track the moving target on the ground stably.

Air–Ground Surveillance Sensor Network based on edge computing for target tracking

With the development of Internet of Things (IoT), Unmanned Aerial Vehicle (UAV) target tracking recently has received lots of attention in research community. The two major research topics in UAV target tracking include loss of a tracked target and high tracking latency, especially in the case where tracking is conducted in an area of many sight-blocking obstacles that put restrictions on a UAV’s flight altitude and block its line of sight. Most recent work on these topics focuses on improving tracking accuracy through adjusting tracking algorithms based on deep learning. But the complexity of these algorithms requires a computation capacity that a regular UAV cannot afford, and therefore tracking failure probability is still non-negligible. To address this challenge, we propose a new target tracking system, referred to as an Air–Ground Surveillance Sensor Network (AGSSN). We build AGSSN by jointly optimizing the network establishment and data transmission, and we design an algorithm ARIT to achieve an optimal tracking performance. We further carry out a series of simulations by deploying our AGSSN on a university’s campus map, and our simulation results show that our proposed AGSSN system can achieve higher reliability and significantly better performance than regular tracking systems in an area with many visually blocking obstacles.

Design of airborne target tracking accelerator based on KCF

The on-board target tracking of unmanned aerial vehicle (UAV) relies on on-board computer to process the collected ground image information, and then use the target tracking algorithm to achieve the target tracking function, which provides the user with target position, trajectory and other information. UAVs are limited by load and power consumption. Due to the high resolution and frame rate of UAV's image collection system, traditional target tracking platforms cannot meet the requirements. As a result, the real-time nature of the tracking platform urgently needs to be resolved. This study designs and implements the airborne ground target tracking accelerator for UAV. The kernelised correlation filters (KCFs) algorithm with low computational complexity and precision is selected to achieve the airborne ground target tracking function and its performance bottleneck is analysed. In order to improve the real-time performance of the tracking task processing, the ZYNQ platform based on ARM and field programmable gate arrays is chosen to achieve the hardware acceleration of the KCF algorithm. The KCF algorithm is implemented in hardware using the high-level synthesis technology, and a high-speed data transmission link is established. Experiments show that the design can achieve a tracking rate of more than 30 FPS under 960 × 540 resolution, which meets the real-time requirements of UAV target tracking task.

Coarse-to-Fine UAV Target Tracking With Deep Reinforcement Learning

The aspect ratio of a target changes frequently during an unmanned aerial vehicle (UAV) tracking task, which makes the aerial tracking very challenging. Traditional trackers struggle from such a problem as they mainly focus on the scale variation issue by maintaining a certain aspect ratio. In this paper, we propose a coarse-to-fine deep scheme to address the aspect ratio variation in UAV tracking. The coarse-tracker first produces an initial estimate for the target object, then a sequence of actions are learned to fine-tune the four boundaries of the bounding box. The coarse-tracker and the fine-tracker are designed to have different action spaces and operating target. The former dominates the entire bounding box and the latter focuses on the refinement of each boundary. They are trained jointly by sharing the perception network with an end-to-end reinforcement learning architecture. Experimental results on benchmark aerial data set prove that the proposed approach outperforms existing trackers and produces significant accuracy gains in dealing with the aspect ratio variation in UAV tracking. Note to Practitioners —During the past years, unmanned aerial vehicle (UAV) have gained much attention for both industrial and consumer uses. It is in urgent demand to endow the UAV with intelligent vision-based techniques, and the automatic target following via visual tracking methods as one of the most fundamental intelligent features could promote various applications of UAVs, such as surveillance, augmented reality, and behavior modeling. Nonetheless, the primary issue of a UAV-based tracking method is the platform itself: it is not stable, it tends to have sudden movements, it generates nonhomogeneous data (scale, angle, rotation, depth, and so on), all of them tend to change the aspect ratio of the target frequently and further increase the difficulty of object tracking. This paper aims to address the aspect ratio change (ARC) problem in UAV tracking. We present a coarse-to-fine strategy for UAV tracking. Specifically, the coarse bounding box is obtained to locate the target firstly. Then, a refinement scheme is performed on each boundary to further improve the position estimate. The tracker is proved to be effective to increase the resistance to the ARC. Such a method can be implemented on UAV to improve the target-following performance.

Shot type constraints in UAV cinematography for autonomous target tracking

During the past years, camera-equipped Unmanned Aerial Vehicles (UAVs) have revolutionized aerial cinematography, allowing easy acquisition of impressive footage. In this context, autonomous functionalities based on machine learning and computer vision modules are gaining ground. During live coverage of outdoor events, an autonomous UAV may visually track and follow a specific target of interest, under a specific desired shot type, mainly adjusted by choosing appropriate focal length and UAV/camera trajectory relative to the target. However, the selected UAV/camera trajectory and the object tracker requirements (which impose limits on the maximum allowable focal length) affect the range of feasible shot types, thus constraining cinematography planning. Therefore, this paper explores the interplay between cinematography and computer vision in the area of autonomous UAV filming. UAV target-tracking trajectories are formalized and geometrically modeled, so as to analytically compute maximum allowable focal length per scenario, to avoid 2D visual tracker failure. Based on this constraint, formulas for estimating the appropriate focal length to achieve the desired shot type in each situation are extracted, so as to determine shot feasibility. Such rules can be embedded into practical UAV intelligent shooting systems, in order to enhance their robustness by facilitating on-the-fly adjustment of the cinematography plan.

An Optimized Image-Based Visual Servo Control for Fixed-Wing Unmanned Aerial Vehicle Target Tracking With Fixed Camera

This paper presents an optimized image-based visual servo (IBVS) control scheme for tracking a ground target by using a fixed-wing unmanned aerial vehicle (UAV) with a monocular camera fixed on it. Unlike the widely used rotor UAVs, the fixed-wing UAV has much more dynamical constraints such as it cannot move omni-direction and its minimum speed is limited by the stalling speed. This makes the target tracking problem more challenging. The proposed scheme leverages the image Jacobian matrix to build a connection between the velocity of the feature point and that of the UAV. Afterward, considering the camera is fixed on the body of the UAV, an “ideal camera” model is proposed to compensate the shifts of the feature point caused by the changes of the UAV's attitude. Then, an optimized control law without solving the pseudo-inverse of the image Jacobian matrix is proposed with the aid of the least square method from the target center in the image coordinate system. This control law takes the velocity of the feature point as inputs and the yaw angular velocity of the UAV as outputs. The stability of the proposed law is analyzed with the Lyapunov method, showing that the UAV will circle around the ground target asymptotically. Finally, the proposed scheme is evaluated by a hardware-in-the-loop (HIL) simulation based on the Gazebo simulator and the off-the-shelf autopilot hardware.

Real-time unmanned aerial vehicle tracking of fast moving small target on ground

To solve the problems of occlusion and fast motion of small targets in unmanned aerial vehicle target tracking, an adaptive algorithm that fuses the improved color histogram tracking response and the correlation filter tracking response based on multichannel histogram of oriented gradient features is proposed to realize small target tracking with high accuracy. The state judgment index is used to determine whether the target is in a fast motion or an occlusion state. In the fast motion state, the search area is enlarged, and the color optimal model that suppresses the suspected area is used for rough detection. Then, redetection in the location of multiple peaks in the rough detection response is carried out using the correlation filter to accurately locate the target. In an occlusion state, the model stops updating, the search area is expanded, and the current color model is used for rough detection. Then, redetection in the place of multiple peaks in the rough detection response is carried out using the correlation filter to accurately locate the target. Experimental results show that the proposed method can track small targets accurately. The frame rate of the proposed method is 40.23 frames/s, indicating usable real-time performance.

Camera Gimbal Control from UAV Autopilot Target Tracking

Unmanned Aerial Vehicle (UAV) is better and safer to carry missions for aerial photography for terrain feature documentation and surveillance. An aerial photography mechanism for camera gimbal control in conjunction with UAV autopilot system is proposed in this paper. A GPS-to-GPS target tracking is implemented into autopilot flight control system to aim at the assigned fixed point target. In order to properly control dynamic photography, coordinate conversion from airborne gimbal frame into the Earth frame via UAV body frame coordinates should be correctly aligned to the target. To avoid gimbal lock problem, the 3D quaternion-based attitude estimation from GPS and IMU is formulated. The UAV autopilot and target tracking control can be implemented on microcontroller with a reliable and stable performance. A UAV autopilot is designed and implemented mounting with 3D gimbal fixture for realization and flight tests. Flight experiment results show good system performance to accomplish the proposed objective.