Aerial Vehicle Tracking Research Articles

Empowering lightweight detectors: Orientation Distillation via anti-ambiguous spatial transformation for remote sensing images

Knowledge distillation (KD) has been one of the most potential methods to implement a lightweight detector, which plays a significant role in satellite in-orbit processing and unmanned aerial vehicle tracking. However, existing distillation paradigms exhibit limited accuracy in detecting arbitrary-oriented objects represented with rotated bounding boxes in remote sensing images. This issue is attributed to two aspects: (i) boundary discontinuity localization distillation, caused by angle periodicity of rotated bounding boxes, and (ii) spatial ossified feature distillation, induced by orientation-agnostic knowledge transitive regions, both of which contribute to ambiguous orientation estimation of objects. To address these issues, we propose an effective KD method called Orientation Distillation (OD) via anti-ambiguous spatial transformation, which consists of two modules. (i) Anti-ambiguous Location Prediction (ALP) module reformulates the regression transformation between teacher–student bounding boxes as Gaussian distributions fitting procedure. These distributions with distilled potential are optimized to accurately localize objects with the aid of boundary continuity cost. (ii) Orientation-guided Feature Calibration (OFC) module employs a learnable affine matrix to augment fixed CNN sampling grid into a spatially remapped one, which bridges between the multi-scale feature of teacher and student for effectively delivering the refined oriented awareness within adaptively distillation regions. Overall, OD customizes the spatial transformation of bounding box representation and sampling grid to transfer anti-ambiguous orientation knowledge, and significantly improves the performance of lightweight detectors upon non-axially arranged objects. Extensive experiments on multiple datasets demonstrate that our plug-and-play distillation framework achieves state-of-the-art performance. Codes are available at https://github.com/Molly6/OD.

Saliency-enhanced background-aware correlation filters with dual temporal regularization for unmanned aerial vehicle tracking

Saliency-enhanced background-aware correlation filters with dual temporal regularization for unmanned aerial vehicle tracking

Lightweight Spatial-Temporal Contextual Aggregation Siamese Network for Unmanned Aerial Vehicle Tracking

Benefiting from the powerful feature extraction capability of deep learning, the Siamese tracker stands out due to its advanced tracking performance. However, constrained by the complex backgrounds of aerial tracking, such as low resolution, occlusion, similar objects, small objects, scale variation, aspect ratio change, deformation and limited computational resources, efficient and accurate aerial tracking is still difficult to realize. In this work, we design a lightweight and efficient adaptive temporal contextual aggregation Siamese network for aerial tracking, which is designed with a parallel atrous module (PAM) and adaptive temporal context aggregation model (ATCAM) to mitigate the above problems. Firstly, by using a series of atrous convolutions with different dilation rates in parallel, the PAM can simultaneously extract and aggregate multi-scale features with spatial contextual information at the same feature map, which effectively improves the ability to cope with changes in target appearance caused by challenges such as aspect ratio change, occlusion, scale variation, etc. Secondly, the ATCAM adaptively introduces temporal contextual information to the target frame through the encoder-decoder structure, which helps the tracker resist interference and recognize the target when it is difficult to extract high-resolution features such as low-resolution, similar objects. Finally, experiments on the UAV20L, UAV123@10fps and DTB70 benchmarks demonstrate the impressive performance of the proposed network running at a high speed of over 75.5 fps on the NVIDIA 3060Ti.

Interframe Saliency Transformer and Lightweight Multidimensional Attention Network for Real-Time Unmanned Aerial Vehicle Tracking

UAV visual-object-tracking technology based on Siamese neural networks has great scientific research and practical application value, and it is widely used in geological surveying, reconnaissance monitoring, and environmental monitoring. Due to the limited onboard computational resources and complex real-world environments of drones, most of the existing tracking systems based on Siamese neural networks struggle to combine excellent performance with high efficiency. Therefore, the key issue is to study how to improve the accuracy of target tracking under the challenges of real-time performance and the above factors. In response to this problem, this paper proposes a real-time UAV tracking system based on interframe saliency transformer and lightweight multidimensional attention network (SiamITL). Specifically, interframe saliency transformer is used to continuously perceive spatial and temporal information, making the network more closely related to the essence of the tracking task. Additionally, a lightweight multidimensional attention network is used to better capture changes in both target appearance and background information, improving the ability of the tracker to distinguish between the target and background. SiamITL is effective and efficient: extensive comparative experiments and ablation experiments have been conducted on multiple aerial tracking benchmarks, demonstrating that our algorithm can achieve more robust feature representation and more accurate target state estimation. Among them, SiamITL achieved success and accuracy rates of 0.625 and 0.818 in the UAV123 benchmark, respectively, demonstrating a certain level of leadership in this field. Furthermore, SiamITL demonstrates the potential for real-time operation on the embedded platform Xavier, highlighting its potential for practical application in real-world scenarios.

Bridging the Gap Between Cumbersome and Light Detectors via Layer-Calibration and Task-Disentangle Distillation in Remote Sensing Imagery

With urgent application requirements such as satellite in-orbit processing and unmanned aerial vehicle tracking, knowledge distillation (KD) following the teacher-student teaching mechanism has shown great potential to obtain lightweight detectors. However, compact students have limited accuracy due to the interference of large scale variations and blurred boundaries in remote sensing objects. Specifically, previous methods mostly force teacher-student responses from the layer of same depth and scale to align. Stereotyped manual inter-layer associations may cause discriminative features of multi-scale objects to be incorrectly bundled. Further, the regression branch follows the identical distillation paradigm as the classification branch, resulting in ambiguous object bounding box deviations. To solve the above two issues, we propose an effective KD framework called layer-calibration and task-disentangle distillation (LTD). First, cross-layer calibration distillation (CCD) structure is innovatively proposed. It adaptively binds a student layer with several related target layers, rather than a fixed layer in the teacher model. Appropriate and clear knowledge of large and small objects is transmitted. Since the CCD structure requires explicit global inner product computation between multiple layers, local implicit calibration (LIC) module is further proposed to reduce distilled convergence difficulty. Second, task-aware spatial disentangle distillation (TASD) structure is devised to transfer task-decoupled semantics and localization knowledge in a divide-and-conquer manner, alleviating objects localization imprecision. Experiments demonstrate that our LTD achieves state-of-the-art performance on several datasets, and is a plug-and-play approach to most detectors. The code will be available soon.

Learning convolutional self-attention module for unmanned aerial vehicle tracking

Learning convolutional self-attention module for unmanned aerial vehicle tracking

Real-Time Tracking Algorithm for Aerial Vehicles Using Improved Convolutional Neural Network and Transfer Learning

To address the poor image feature extraction ability, excessive tracking time, and low accuracy of traditional real-time algorithms for aerial tracking of vehicle traffic, a real-time tracking algorithm using improved convolutional neural network (CNN) and transfer learning was proposed herein. In this algorithm, first, the aerial vehicle images are matched with sample images for the image offset calibration. Second, the CNN parameters are initialized by constructing a filter set, and transfer learning is employed to construct a CNN pre-training model. Third, a deep convolution feature extraction structure map is designed to extract the depth features of images. Finally, based on the depth features, the target vehicle motion model is established, the similarity between the target and candidate models is calculated, and the real-time tracking of aerial vehicles is completed. The results show that the image correction accuracy of the proposed algorithm is as high as 92%. The algorithm yields satisfactory results in terms of feature extraction and calculation accuracy. Moreover, it has a small overall error, the average tracking time required by it is only 22.8 s, and its false negative rate is as low as 0.4%. Therefore, the proposed algorithm has considerable potential practical application.

The Recent Advancement in Unmanned Aerial Vehicle Tracking Antenna: A Review.

Unmanned aerial vehicle (UAV) antenna tracking system is an electromechanical component designed to track and steer the signal beams from the ground control station (GCS) to the airborne platform for optimum signal alignment. In a tracking system, an antenna continuously tracks a moving target and records their position. A UAV tracking antenna system is susceptible to signal loss if omnidirectional antenna is deployed as the preferred design. Therefore, to achieve longer UAV distance communication, there is a need for directional high gain antenna. From design principle, directional antennas are known to focus their signal energy in a particular direction viewed from their radiation pattern which is concentrated in a particular azimuth direction. Unfortunately, a directional antenna is limited by angle, thus, it must always be directed to the target. The other limitation of a UAV mechanical beam steering system is that the system is expensive to maintain and with low reliability. To solve this problem, we are proposing the use of MIMO technology as a readily available technology for UAV beyond line of sight technology. Although UAV antenna tracking is domiciled in the mechanical beam steering arrangement, this study shows that this native technology could be usurped by MIMO beam forming.

Quaternion‐based adaptive control for trajectory tracking of quadrotor unmanned aerial vehicles

SummaryThe problem of trajectory tracking of aerial vehicles with four rotors and dynamic parameter uncertainties is addressed in this article. By exploiting the cascade structure of the translational and attitude dynamics of the quadrotor, a nonlinear hierarchical adaptive control is proposed. The attitude control law is designed based on the unit quaternion, therefore, the singularities of the Euler angles and angle‐axis representation are avoided. Instead of relying on the stability properties of time‐varying cascade systems, the stability of the whole closed‐loop system is proven by an appropriate Lyapunov function. In addition, a novel method to compute the desired orientation based on the desired Cartesian trajectories and position control inputs is proposed. A set of simulations results are presented to assess the performance of the proposed adaptive controller.

UAV tracking based on saliency detection

This paper presents a novel unmanned aerial vehicle tracking framework. First, hierarchical convolutional neural network features are used to track the object independently in a correlation filter tracking framework. Second, a stability criterion is proposed, which is based on the variance of tracking results of each layer. Next, tracking result is adaptively fused via the variance. Meanwhile, the criterion can be used to measure the quality of tracking results. A saliency detection method is utilized to generate candidate regions when tracking failure occurs. By virtue of this method, our tracking algorithm can robustly cope with appearance changes and prevent drifting issues. Experimental results show that our proposed tracking algorithm performs favorably against state-of-the-art methods on two benchmark datasets.

Incorporating Trust and Self-Confidence Analysis in the Guidance and Control of (Semi)Autonomous Mobile Robotic Systems

We propose a trust and self-confidence-based autonomy allocation strategy to automatically choose between manual and autonomous control of (semi)autonomous mobile robots in guidance and navigation tasks. We utilize a performance-centric, computational trust and self-confidence model and automated autonomy allocation strategy, developed in our earlier work (H. Saeidi and Y. Wang, “Trust and self-confidence based autonomy allocation for robotic systems,” in Proc. 54th IEEE Conf. Decis. Control , 2015, pp. 6052–6057.), based on objective and unbiased performance measures for the human and the robot. A set of robot simulations with a human-in-the-loop is conducted for a teleoperated unmanned aerial vehicle tracking task. The results demonstrate that our allocation strategy can capture human autonomy allocation pattern with an accuracy of 64.05%. We also show that the strategy can improve the overall robot performance by 11.76% and reduce operator's workload by 10.07% compared to a manual allocation. Moreover, compared to a performance maximization strategy, our strategy is 23.42% more likely to be accepted and generally preferred and trusted by the participants. Furthermore, we design a decision pattern correction algorithm based on nonlinear model predictive control to help a human operator gradually adapt to a modified allocation pattern for improved overall performance.

Tracking in Aerial Hyperspectral Videos Using Deep Kernelized Correlation Filters

Hyperspectral imaging holds enormous potential to improve the state of the art in aerial vehicle tracking with low spatial and temporal resolutions. Recently, adaptive multimodal hyperspectral sensors have attracted growing interest due to their ability to record extended data quickly from aerial platforms. In this paper, we apply popular concepts from traditional object tracking, namely, kernelized correlation filters (KCFs) and deep convolutional neural network features to aerial tracking in the hyperspectral domain. We propose the deep hyperspectral KCF-based tracker (DeepHKCF) to efficiently track aerial vehicles using an adaptive multimodal hyperspectral sensor. We address low temporal resolution by designing a single KCF-in-multiple regions-of-interest (ROIs) approach to cover a reasonably large area. To increase the speed of deep convolutional features extraction from multiple ROIs, we design an effective ROI mapping strategy. The proposed tracker also provides flexibility to couple with the more advanced correlation filter trackers. The DeepHKCF tracker performs exceptionally well with deep features set up in a synthetic hyperspectral video generated by the digital imaging and remote sensing image generation (DIRSIG) software. In addition, we generate a large, synthetic, single-channel data set using DIRSIG to perform vehicle classification in the wide-area motion imagery (WAMI) platform. This way, the high fidelity of the DIRSIG software is proven, and a large-scale aerial vehicle classification data set is released to support studies on vehicle detection and tracking in the WAMI platform.

Drone-Based High-Resolution Tracking of Aquatic Vertebrates

Determining the small-scale movement patterns of marine vertebrates usually requires invasive active acoustic tagging or in-water monitoring, with the inherent behavioural impacts of those techniques. In addition, these techniques rarely allow direct continuous behavioural assessments or the recording of environmental interactions, especially for highly mobile species. Here, we trial a novel method of assessing small-scale movement patterns of marine vertebrates using an unmanned aerial vehicle that could complement longer-term tracking approaches. This approach is unlikely to have behavioural impacts and provides high accuracy and high frequency location data (10 Hz), while subsequently allowing quantitative trajectory analysis. Unmanned aerial vehicle tracking is also relatively low cost compared to single-use acoustic and GPS tags. We tracked 14 sharks for up to 10 min in a shallow lagoon of Heron Island, Australia. Trajectory analysis revealed that Epaulette sharks (Hemiscyllium ocellatum) displayed sinusoidal movement patterns, while Blacktip Reef Sharks (Carcharhinus melanopterus) had more linear trajectories that were similar to those of a Lemon shark (Negaprion acutidens). Individual shark trajectory patterns and movement speeds were highly variable. Results indicate that Epaulette sharks may be more mobile during diurnal low tides than previously thought. The approach presented here allows the movements and behaviours of marine vertebrates to be analysed at resolutions not previously possible without complex and expensive acoustic arrays. This method would be useful to assess the habitat use and behaviours of sharks and rays in shallow water environments, where they are most likely to interact with humans.

Safe map generation after a disaster, assisted by an unmanned aerial vehicle tracking system

This paper proposes a new method to build safe maps through disaster‐stricken zones by taking advantage of different scenarios of tracking pedestrians and area scanning. Tracking is usually achieved by following the refugees' mobile devices. However, sometimes refugees do not hold mobiles to track them. To overcome this problem, we exploit the tracking systems of unmanned aerial vehicles (UAVs), which enable both image processing and mobile tracking depending on the scenario. UAV technology is a low‐cost, flexible solution in missions that are difficult to execute by humans and other systems. After a disaster event, a safe map can be constructed from aerial imagery data of the moving pedestrians. Aerial data are also useful for evaluating damage in the stricken areas. The present study integrates UAVs into the Internet of Things, generating a pseudo scenario for the map generation. Experiments confirmed the usefulness of low‐cost UAVs in tracking and scanning with and without smartphone tracking data. © 2018 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Real-time unmanned aerial vehicle tracking of fast moving small target on ground

To solve the problems of occlusion and fast motion of small targets in unmanned aerial vehicle target tracking, an adaptive algorithm that fuses the improved color histogram tracking response and the correlation filter tracking response based on multichannel histogram of oriented gradient features is proposed to realize small target tracking with high accuracy. The state judgment index is used to determine whether the target is in a fast motion or an occlusion state. In the fast motion state, the search area is enlarged, and the color optimal model that suppresses the suspected area is used for rough detection. Then, redetection in the location of multiple peaks in the rough detection response is carried out using the correlation filter to accurately locate the target. In an occlusion state, the model stops updating, the search area is expanded, and the current color model is used for rough detection. Then, redetection in the place of multiple peaks in the rough detection response is carried out using the correlation filter to accurately locate the target. Experimental results show that the proposed method can track small targets accurately. The frame rate of the proposed method is 40.23 frames/s, indicating usable real-time performance.

A unitized guidance law for over-flight and standoff tracking of a ground target with fixed wing unmanned aerial vehicle

Over-flight and standoff tracking are the two typical modes of fixed wing unmanned aerial vehicle tracking ground targets. The guidance law of each mode is usually designed independently and thus shows different forms. In this paper, based on the constraints on unmanned aerial vehicle flight, a novel unitized guidance law, which is compatible with these two modes, is proposed. The stability of the guidance law is rigorously proved by a Lyapunov function, and the design of guidance parameters is discussed. Numerical simulation results show that the proposed guidance law can be used for automatically tracking the stationary and moving ground targets well, and also can switch between the two tracking modes smoothly.

Test bed for applications of heterogeneous unmanned vehicles

This article addresses the development and implementation of a test bed for applications of heterogeneous unmanned vehicle systems. The test bed consists of unmanned aerial vehicles (Parrot AR.Drones versions 1 or 2, Parrot SA, Paris, France, and Bebop Drones 1.0 and 2.0, Parrot SA, Paris, France), ground vehicles (WowWee Rovio, WowWee Group Limited, Hong Kong, China), and the motion capture systems VICON and OptiTrack. Such test bed allows the user to choose between two different options of development environments, to perform aerial and ground vehicles applications. On the one hand, it is possible to select an environment based on the VICON system and LabVIEW (National Instruments) or robotics operating system platforms, which make use the Parrot AR.Drone software development kit or the Bebop_autonomy Driver to communicate with the unmanned vehicles. On the other hand, it is possible to employ a platform that uses the OptiTrack system and that allows users to develop their own applications, replacing AR.Drone’s original firmware with original code. We have developed four experimental setups to illustrate the use of the Parrot software development kit, the Bebop Driver (AutonomyLab, Simon Fraser University, British Columbia, Canada), and the original firmware replacement for performing a strategy that involves both ground and aerial vehicle tracking. Finally, in order to illustrate the effectiveness of the developed test bed for the implementation of advanced controllers, we present experimental results of the implementation of three consensus algorithms: static, adaptive, and neural network, in order to accomplish that a team of multiagents systems move together to track a target.

Integrating Hyperspectral Likelihoods in a Multidimensional Assignment Algorithm for Aerial Vehicle Tracking

Tracking vehicles through dense environments is an important and challenging task that is mostly tackled using visible and near IR wavelengths. Hyperspectral imaging is known to improve the robustness of target identification, but the massive increase in data created is usually prohibitive for tracking many targets. We present a persistent real-time aerial target tracking system, taking advantage of an adaptive, multimodal sensor concept and blending the hyperspectral likelihoods with kinematic likelihoods in a multidimensional assignment framework. The adaptive sensor is capable of providing wide field of view panchromatic images as well as the spectra of small number of pixels. The proposed system does not require large amount of hyperspectral data collection as we focus on tracking fewer number of targets with higher persistency. This overcomes the data challenge of hyperspectral tracking by following dynamic data-driven application systems (DDDAS) principles to control hyperspectral data collection where most beneficial. The DDDAS framework for controlling hyperspectral data collection is developed by incorporating prior information from the filter movement predictions and information from motion detection. The proposed multidimensional hyperspectral feature-aided tracker is compared to a 2-D hyperspectral feature-aided tracker and another cascaded hyperspectral data based tracker by generating a synthetic, realistic, aerial video on a dense scene.

Frequency-dependent beam pattern of acoustic arrays for unmanned aerial vehicle tracking

There are practical constraints on the geometry and construction of acoustic arrays which are designed to detect small unmanned aerial vehicles (UAVs). The constraints depend to a large degree on the environment in which the array must operate. For example, an acoustic array which provides UAV detection to protect a public stadium is likely to differ considerably from one which soldiers can carry into the field and deploy to protect their position. The former can use almost unlimited power, be very large and be composed of many elements, while the latter will probably run on batteries or solar and must be man-portable. These different designs provide different performance as measured by detection range, array gain, and angular resolution, for example. This talk presents the performance of several common array configuration over the band of frequencies in which passive AUV detection at tactically useful ranges is possible.

A Study on the Multiple UAVs Cooperative Fire Fighting based on Consensus Algorithm

As for multi-uninhabited aerial vehicle cooperative fire fighting problem, a multi-uninhabited aerial vehicle cooperative control method was proposed based on consensus algorithm. multi-uninhabited aerial vehicle cooperative control problem was divided into two steps: 1) a multi-uninhabited aerial vehicle arrival simultaneous control strategy under influence of wind field based on consensus algorithm was presented, which achieves the target of multi-uninhabited aerial vehicle arrival simultaneous at fire front and scattered around the fire front, and prepared for cooperative fire fighting; 2) Applying consensus algorithm, cooperative tracking of the designed uninhabited aerial vehicle motional orbit, and fighting fire from outside to inside to avoid recrudescence. In the process of multi-uninhabited aerial vehicle tracking fire front and fire fighting, both step considered the influence of wind, achieved multi-UAV fighting fire simultaneous, shortened fire fighting time to decreased fire loss to minimum. Simulation results demonstrated that the proposed method could achieve target of multi-uninhabited aerial vehicle fire fighting effectively.