Real Unmanned Aerial Vehicles Research Articles

Suboptimal Trajectory Planning Technique in Real UAV Scenarios with Partial Knowledge of the Environment

In recent years, the issue of trajectory planning for autonomous unmanned aerial vehicles (UAVs) has received significant attention due to the rising demand for these vehicles across various applications. Despite advancements, real-time trajectory planning remains computationally demanding, particularly with the inclusion of 3D localization using computer vision or advanced sensors. Consequently, much of the existing research focuses on semi-autonomous systems, which rely on ground assistance through the use of external sensors (motion capture systems) and remote computing power. This study addresses the challenge by proposing a fully autonomous trajectory planning solution. By introducing a real-time path planning algorithm based on the minimization of the snap, the optimal trajectory is dynamically recalculated as needed. Evaluation of the algorithm’s performance is conducted in an unknown real-world scenario, utilizing both simulations and experimental data. The algorithm was implemented in MATLAB and subsequently translated to C++ for onboard execution on the drone.

A Real-time Fuzzy Interacting Multiple-Model Velocity Obstacle Avoidance Approach for Unmanned Aerial Vehicles

This paper presents a new fuzzy interacting multiple-model velocity obstacle (FIMVO) approach for collision avoidance of unmanned aerial vehicles (UAVs). The proposed approach adopts in one framework the advantages of geometric collision avoidance approaches, namely of the velocity (VO), reciprocal velocity (RVO), and hybrid reciprocal velocity obstacle (HRVO) avoidance approaches combined with fuzzy logic. This leads to a combined decision-making rule, with real-time efficiency. The developed approach is compared with geometric conventional velocity obstacle avoidance approaches: VO, RVO, and HRVO avoidance approaches. The proposed approach is carefully evaluated and validated in a simulation environment and over real UAVs. The case study includes three mini UAVs and a human teleoperator who can control only one of them. The other UAVs used the computer-based teleoperator with the proposed and compared approaches. The performance criteria have been defined in four parts: trajectory smoothness, task performance, algorithm simplicity, and reliability. In 1000 independently repeated simulations, the performance results showed that the proposed FIMVO approach was 10 times better than the VO approach in terms of the number of avoided collisions. The statistical analysis demonstrates that the proposed FIMVO approach outperforms geometric velocity obstacle avoidance approaches concerning reliability and real-time efficiency.

Detection of Military Targets on Ground and Sea by UAVs with Low-Altitude Oblique Perspective

Small-scale low-altitude unmanned aerial vehicles (UAVs) equipped with perception capability for military targets will become increasingly essential for strategic reconnaissance and stationary patrols in the future. To respond to challenges such as complex terrain and weather variations, as well as the deception and camouflage of military targets, this paper proposes a hybrid detection model that combines Convolutional Neural Network (CNN) and Transformer architecture in a decoupled manner. The proposed detector consists of the C-branch and the T-branch. In the C-branch, Multi-gradient Path Network (MgpNet) is introduced, inspired by the multi-gradient flow strategy, excelling in capturing the local feature information of an image. In the T-branch, RPFormer, a Region–Pixel two-stage attention mechanism, is proposed to aggregate the global feature information of the whole image. A feature fusion strategy is proposed to merge the feature layers of the two branches, further improving the detection accuracy. Furthermore, to better simulate real UAVs’ reconnaissance environments, we construct a dataset of military targets in complex environments captured from an oblique perspective to evaluate the proposed detector. In ablation experiments, different fusion methods are validated, and the results demonstrate the effectiveness of the proposed fusion strategy. In comparative experiments, the proposed detector outperforms most advanced general detectors.

A systematic approach for threat and vulnerability analysis of unmanned aerial vehicles

The proliferation of Unmanned Aerial Vehicles (UAVs) is expected to experience a substantial rise in the next years, driven by their ever-increasing application across various domains. However, ensuring secure communication between UAVs and their ground stations is crucial to prevent the unauthorized disclosure of sensitive information that could jeopardize the mission’s integrity when exploited by malicious actors. Despite this importance, several UAV systems currently operate based on open-source command and control technologies, which have overlooked several security considerations while focusing on availability and safety. To address this concern, this study conducts a comprehensive security assessment of UAV-based systems starting with a systematic literature review whose main purpose is building a comprehensive catalog of threats associated with this technology. Particular attention has been paid to the MAVlink protocol, an open-source protocol commonly utilized for telemetry and command and control for multiple UAVs. Therefore, drawing upon the built catalog, a threat modeling and penetration testing technique has been employed to examine the MAVlink implementation on a real UAV. A threat model developed for a specific case study is also presented, leading to the discovery of four new vulnerabilities, some of which were successfully exploited through attacks. By shedding light on these vulnerabilities, this work seeks to encourage further investigation and research to develop robust security mechanisms for UAV communication systems. It is imperative to address these vulnerabilities proactively to enhance the overall security posture and safeguard against potential threats in the UAV ecosystem.

Predefined-time control design for tracking chaotic trajectories around a contour with an UAV

The surveillance or monitoring of places is crucial to ensuring security, protecting people and assets, preventing crimes, and detecting emergencies, to mention some. Unmanned Aerial Vehicles (UAVs) play a vital role in these applications, offering versatility, agility, and aerial vision. A crucial step for such tasks is to protect the UAV path ahead. This paper focuses on a methodology harnessing the unpredictable nature of chaotic systems to generate trajectories around a closed area or contour. However, although a vast quantity of research papers mention the use of chaotic path generation, they have yet to learn about the control system and the dynamics affecting the UAV, where developing the control theory is challenging. In this paper, we design controllers based on predetermined-time stability, ensuring the achievement of the desired trajectory before a specified time. Additionally, adjusting control parameters is a crucial step during the control design, impacting the control performance. Hence, we present a method to optimize and adapt controller parameters through evolutionary optimization, demonstrating precision enhancement. We validate the proposed system’s performance and the controllers through numerical simulations, indicating that the UAV effectively and accurately follows some types of chaotic trajectories like a square contour, aiming at the feasibility of this methodology in real UAV surveillance applications.

Development of Real-Time Unmanned Aerial Vehicle Urban Object Detection System with Federated Learning

In this paper, an urban object detection system via unmanned aerial vehicles (UAVs) is developed to collect real-time traffic information, which can be further utilized in many applications such as traffic monitoring and urban traffic management. The system includes an object detection algorithm, deep learning model training, and deployment on a real UAV. For the object detection algorithm, the Mobilenet-SSD model is applied owing to its lightweight and efficiency, which make it suitable for real-time applications on an onboard microprocessor. For model training, federated learning (FL) is used to protect privacy and increase efficiency with parallel computing. Last, the FL-trained object detection model is deployed on a real UAV for real-time performance testing. The experimental results show that the object detection algorithm can reach a speed of 18 frames per second with good detection performance, which shows the real-time computation ability of a resource-limited edge device and also validates the effectiveness of the developed system.

Data-driven multivariate regression-based anomaly detection and recovery of unmanned aerial vehicle flight data

Abstract Flight data anomaly detection is crucial to ensuring the safe operation of unmanned aerial vehicles (UAVs) and has been extensively studied. However, the accurate modeling and analysis of flight data is challenging due to the influence of random noise. Meanwhile, existing methods are often inadequate in parameter selection and feature extraction when dealing with large-scale and high-dimensional flight data. This paper proposes a data-driven multivariate regression-based framework considering spatio-temporal correlation for UAV flight data anomaly detection and recovery, which integrates the techniques of correlation analysis (CA), one-dimensional convolutional neural network and long short-term memory (1D CNN-LSTM), and error filtering (EF), named CA-1DCL-EF. Specifically, correlation analysis is first performed on original UAV flight data to select parameters with correlation to reduce the model input and avoid the negative impact of irrelevant parameters on the model. Next, a regression model based on 1D CNN-LSTM is designed to fully extract the spatio-temporal features of UAV flight data and realize parameter mapping. Then, to overcome the effect of random noise, a filtering technique is introduced to smooth the errors to improve the anomaly detection performance. Finally, two common anomaly types are injected into real UAV flight datasets to verify the effectiveness of the proposed method.

Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments.

Attitude control of fixed-wing unmanned aerial vehicles (UAVs) is a difficult control problem in part due to uncertain nonlinear dynamics, actuator constraints, and coupled longitudinal and lateral motions. Current state-of-the-art autopilots are based on linear control and are thus limited in their effectiveness and performance. Gls drl is a machine learning method to automatically discover optimal control laws through interaction with the controlled system that can handle complex nonlinear dynamics. We show in this article that deep reinforcement learning (DRL) can successfully learn to perform attitude control of a fixed-wing UAV operating directly on the original nonlinear dynamics, requiring as little as 3 min of flight data. We initially train our model in a simulation environment and then deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller with no further online learning required. Learning with significant actuation delay and diversified simulated dynamics were found to be crucial for successful transfer to control of the real UAV. In addition to a qualitative comparison with the ArduPlane autopilot, we present a quantitative assessment based on linear analysis to better understand the learning controller's behavior.

UAV photogrammetry for red tide monitoring and geo-localization via onboard GNSS/IMU data

As one of the worst ecological disasters in the world’s oceans, red tide seriously threatens marine ecology. Red tide monitoring is essential for preserving the ocean ecosystem. However, traditional photogrammetric processing workflows, e.g. SfM (Structure from Motion) and MVS (Multi-view Stereo) based image orientation and dense matching, do not apply to offshore images due to the low texture of ocean color. The primary contribution of this study is a UAV (Unmanned Aerial Vehicle)-based photogrammetric solution for red tide monitoring and direct geo-localization. First, a direct geo-localization model has been created, which solely uses onboard sensor data for the 3D coordinate calculation of 2D image targets based on the collinear equation. Second, two UAVs are chosen as photogrammetric systems that can supply the necessary data for the direct geo-localization model, including high-resolution images, camera intrinsic and extrinsic parameters. Third, a human annotation technique has been utilized to identify red tide regions whose ground polygons can be further estimated using the direct geo-localization model, taking into account how difficult it is to detect red tides automatically. Finally, the accuracy of direct geo-localization and the viability of the proposed solution has been evaluated and verified using real UAV datasets. The experimental results show that the direct geo-localization precision is better than 20 m when only onboard sensor data is used. The suggested workflow is appropriate for monitoring red tides, which can provide critical information for managing marine ecosystems. The executable toolkit would be made publicly available at https://github.com/json87/PhotoDigitizer.

Method for Determining Coastline Course Based on Low-Altitude Images Taken by a UAV

In recent years, the most popular methods for determining coastline course are geodetic, satellite, and tacheometric techniques. None of the above-mentioned measurement methods allows marking out the shoreline both in an accurate way and with high coverage of the terrain with surveys. For this reason, intensive works are currently underway to find alternative solutions that could accurately, extensively, and quickly determine coastline course. Based on a review of the literature regarding shoreline measurements, it can be concluded that the photogrammetric method, based on low-altitude images taken by an Unmanned Aerial Vehicle (UAV), has the greatest potential. The aim of this publication is to present and validate a method for determining coastline course based on low-altitude photos taken by a drone. Shoreline measurements were carried out using the DJI Matrice 300 RTK UAV in the coastal zone at the public beach in Gdynia (Poland) in 2023. In addition, the coastline course was marked out using high-resolution satellite imagery (0.3–0.5 m). In order to calculate the accuracy of determining the shoreline by photogrammetric and satellite methods, it was decided to relate them to the coastline marked out using a Global Navigation Satellite System (GNSS) Real Time Kinematic (RTK) receiver with an accuracy of 2.4 cm Distance Root Mean Square (DRMS). Studies have shown that accuracies of determining coastline course using a UAV are 0.47 m (p = 0.95) for the orthophotomosaic method and 0.70 m (p = 0.95) for the Digital Surface Model (DSM), and are much more accurate than the satellite method, which amounted to 6.37 m (p = 0.95) for the Pléiades Neo satellite and 9.24 m (p = 0.95) for the Hexagon Europe satellite. Based on the obtained test results, it can be stated that the photogrammetric method using a UAV meets the accuracy requirements laid down for the most stringent International Hydrographic Organization (IHO) order, i.e., Exclusive Order (Total Horizontal Uncertainty (THU) of 5 m with a confidence level of 95%), which they relate to coastline measurements.

Bridging the Gap between Simulation and Real Autonomous UAV Flights in Industrial Applications

The utilization of autonomous unmanned aerial vehicles (UAVs) has increased rapidly due to their ability to perform a variety of tasks, including industrial inspection. Conducting testing with actual flights within industrial facilities proves to be both expensive and hazardous, posing risks to the system, the facilities, and their personnel. This paper presents an innovative and reliable methodology for developing such applications, ensuring safety and efficiency throughout the process. It involves a staged transition from simulation to reality, wherein various components are validated at each stage. This iterative approach facilitates error identification and resolution, enabling subsequent real flights to be conducted with enhanced safety after validating the remainder of the system. Furthermore, this article showcases two use cases: wind turbine inspection and photovoltaic plant inspection. By implementing the suggested methodology, these applications were successfully developed in an efficient and secure manner.

LSVL: Large-scale season-invariant visual localization for UAVs

Localization of autonomous unmanned aerial vehicles (UAVs) relies heavily on Global Navigation Satellite Systems (GNSS), which are susceptible to interference. Especially in security applications, robust localization algorithms independent of GNSS are needed to provide dependable operations of autonomous UAVs also in interfered conditions. Typical non-GNSS visual localization approaches rely on known starting pose, work only on a small-sized map, or require known flight paths before a mission starts. We consider the problem of localization with no information on initial pose or planned flight path. We propose a solution for global visual localization on large maps, based on matching orthoprojected UAV images to satellite imagery using learned season-invariant descriptors, and test with environment sizes up to 100 km2. We show that the method is able to determine heading, latitude and longitude of the UAV at 12.6–18.7 m lateral translation error in as few as 23.2–44.4 updates from an uninformed initialization, also in situations of significant seasonal appearance difference (winter–summer) between the UAV image and the map. We evaluate the characteristics of multiple neural network architectures for generating the descriptors, and likelihood estimation methods that are able to provide fast convergence and low localization error. We also evaluate the operation of the algorithm using real UAV data and evaluate running time on a real-time embedded platform. We believe this is the first work that is able to recover the pose of an UAV at this scale and rate of convergence, while allowing significant seasonal difference between camera observations and map.

Precise crop classification of UAV hyperspectral imagery using kernel tensor slice sparse coding based classifier

Precise crop classification plays a significant role in the agriculture field. An appropriate data source for precise crop classification is high spatial resolutions hyperspectral imagery (H2 imagery) acquired by unmanned aerial vehicle (UAV). However, for imagery with many different classes of crops, crop classification of UAV H2 imagery is a huge challenge. The significant spectral diversity, spatial heterogeneity and nonlinear data structure of UAV H2 imagery results in poor spectral discriminability. To improve the discriminability, a kernel tensor slice sparse coding-based classifier (KTSSCC) is proposed for precise crop classification of UAV H2 imagery in this research. The kernel tensor representation mechanism in KTSSCC can reduce the nonlinear separation while well preserving the spectral characteristics and spatial constraints of land-covers, and thus the discriminability is greatly improved. Furthermore, this paper puts forward the kernel tensor slice sparse orthogonal matching pursuit (KTSSOMP) algorithm to optimize kernel tensor slice sparse coding in the spectral space, which greatly reduces the computation cost. Moreover, there are very few parameters to be tuned in our proposed model. We assess the performance of KTSSCC on two real UAV hyperspectral imagery datasets, and find that, based on visual and quantitative results, it provides satisfactory crop classification results and outperforms the state-of-the-art approaches.

Template-guided frequency attention and adaptive cross-entropy loss for UAV visual tracking

This paper addresses the problem of visual object tracking for Unmanned Aerial Vehicles (UAVs). Most Siamese trackers are used to regard object tracking as classification and regression problems. However, it is difficult for these trackers to accurately classify in the face of similar objects, background clutters and other common challenges in UAV scenes. So, a reliable classifier is the key to improving UAV tracking performance. In this paper, a simple yet efficient tracker following the basic architecture of the Siamese neural network is proposed, which improves the classification ability from three stages. First, the frequency channel attention module is introduced to enhance the target features via frequency domain learning. Second, a template-guided attention module is designed to promote information exchange between the template branch and the search branch, which can get reliable classification response maps. Third, adaptive cross-entropy loss is proposed to make the tracker focus on hard samples that contribute more to the training process, solving the data imbalance between positive and negative samples. To evaluate the performance of the proposed tracker, comprehensive experiments are conducted on two challenging aerial datasets, including UAV123 and UAVDT. Experimental results demonstrate that the proposed tracker achieves favorable tracking performances in aerial benchmarks beyond 41 frames/s. We conducted experiments in real UAV scenes to further verify the efficiency of our tracker in the real world.

Unmanned aerial vehicle strike on a flat plate: Tests and numerical simulations

The incursion of Unmanned Aerial Vehicles (UAVs) into airports often occurs due to the popularity of drones, which may lead to a threat to aircraft flight safety. Therefore, estimating the dynamic impact load caused by drone strikes is essential. This paper proposes a test method with high precision and low cost involving launching of a UAV to impact a flat plate specimen by using an air gun. The test results of UAVs impacting flat plates at different impact velocities, such as the UAV damage deformation captured by a high-speed camera and strain vs time dynamic response curves of plates, were obtained and analysed. At the same time, a corresponding numerical simulation was carried out by using the explicit finite element software LS-DYNA. The predicted damage to the UAV and strain on the flat plate during the strike process were compared with the test results. The overall trend of the simulation results is in good agreement with the test results, at least for the first three milliseconds of the event. This shows that the numerical simulation model established in this paper is reasonable. The UAV numerical method established in the present paper can be used to carry out numerical simulations and evaluations of the collision safety of UAVs against large aircraft and high-value ground targets. The results show that the local deformation of the impacted target is uneven due to the distribution of concentrated mass components such as motors, battery, and camera. As the impact velocity of the UAV increases, all parts of the UAV are seriously damaged and basically in a fragmented state, and the battery is greatly deformed. The interaction between the UAV and the flat plate specimen is approximately 2.7 ms, and the UAV numerical simulation model established in this paper can well simulate the real UAV impact process.

SPINS: A structure priors aided inertial navigation system

AbstractWe propose a navigation system combining sensor‐aided inertial navigation and prior‐map‐based localization to improve the stability and accuracy of robot localization in structure‐rich environments. Specifically, we adopt point, line, and plane features in the navigation system to enhance the feature richness in low‐texture environments and improve the localization reliability. We additionally integrate structure prior information of the environments to constrain the localization drifts and improve the accuracy. The prior information is called structure priors and parameterized as low‐dimensional relative distances/angles between different geometric primitives. The localization is formulated as a graph‐based optimization problem that contains sliding‐window‐based variables and factors, including Inertial Measurement Unit, heterogeneous features, and structure priors. A limited number of structure priors are selected based on the information gain to alleviate the computation burden. Finally, the proposed framework is extensively tested on synthetic data, public data sets, and, more importantly, on the real Unmanned Aerial Vehicle flight data obtained from both indoor and outdoor inspection tasks. The results show that the proposed scheme can effectively improve the accuracy and robustness of localization for autonomous robots in civilian applications.

Energy Efficient Path Planning for 3D Aerial Inspections

The use of Unmanned Aerial Vehicles (UAVs) in different inspection tasks is increasing. This technology reduces inspection costs and collects high quality data of distinct structures, including areas that are not easily accessible by human operators. However, the reduced energy available on the UAVs limits their flight endurance. To increase the autonomy of a single flight, it is important to optimize the path to be performed by the UAV, in terms of energy loss. Therefore, this work presents a novel formulation of the Travelling Salesman Problem (TSP) and a path planning algorithm that uses a UAV energy model to solve this optimization problem. The novel TSP formulation is defined as Asymmetric Travelling Salesman Problem with Precedence Loss (ATSP-PL), where the cost of moving the UAV depends on the previous position. The energy model relates each UAV movement with its energy consumption, while the path planning algorithm is focused on minimizing the energy loss of the UAV, ensuring that the structure is fully covered. The developed algorithm was tested in both simulated and real scenarios. The simulated experiments were performed with realistic models of wind turbines and a UAV, whereas the real experiments were performed with a real UAV and an illumination tower. The inspection paths generated presented improvements over 24% and 8%, when compared with other methods, for the simulated and real experiments, respectively, optimizing the energy consumption of the UAV.

MAda-Net: Model-Adaptive Deep Learning Imaging for SAR Tomography

The compressive sensing (CS)-based tomographic SAR (TomoSAR) 3-D imaging method has the shortcoming of low efficiency, mainly represented in two aspects: first, the CS solver requires iterative calculation and hence is computationally expensive; second, the CS solver needs hyperparameters’ selection, which commonly requires cost-inefficient try-and-error attempts. Recently, the iterative CS solver is suggested to be replaced by a deep learning network for a tremendous processing speed improvement. However, the existing deep-learning-based TomoSAR imaging algorithms suffer from the problem of model inadaptability, i.e., being inadaptive to the observation model and the signal energy model and hence is low accuracy. This article proposes a new model-adaptive network (MAda-Net) to implement deep-learning-based TomoSAR 3-D imaging with a much improved processing accuracy. First, a new adaptive model-solving (AMS) module is introduced to solve the problem of the observation model inconsistency between the real spatially varying one and the approximately fixed one used by the network. Second, a new adaptive threshold-activation (ATC) module is introduced to solve the problem of signal energy model inconsistency between the real backscattered echo and the simulated echo for network training. The effectiveness of the proposed method has been verified by the computer simulations and the real unmanned aerial vehicle (UAV) SAR experiments.

Rotor Noise-Aware Noise Covariance Matrix Estimation for Unmanned Aerial Vehicle Audition

A noise covariance matrix (NCM) estimation method for unmanned aerial vehicle (UAV) audition is proposed with rotor noise reduction as its primary focus. The proposed NCM estimation method could be incorporated into audio processing algorithms using UAV-mounted microphone array systems. The NCM is formed through accurate estimation of the microphone array input signal's amplitude and phase by using a multi-sensory rotor noise power spectral density (PSD) estimator and a filter formed by exploiting the acoustical relationship between the microphone array and the rotor noise sources, respectively. The estimated NCM aims to be readily incorporable into several source enhancement algorithms to reduce the effects of rotor noise and improve the resultant audio quality. Experiment evaluation using real in-flight UAV rotor noise recordings shows that the estimated NCM significantly improves rotor noise reduction (of up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim 28$</tex-math></inline-formula> dB) and the quality of the target sound.

Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environments

Unmanned aerial vehicles (UAVs) have been widely used in military warfare, and realizing safely autonomous motion control (AMC) in complex unknown environments is a challenge to face. In this paper, we formulate the AMC problem as a Markov decision process (MDP) and propose an advanced deep reinforcement learning (DRL) method that allows UAVs to execute complex tasks in different environments. Aiming to overcome the limitations of the prioritized experience replay (PER), the proposed asynchronous curriculum experience replay (ACER) uses multithreads to asynchronously update the priorities and assigns the true priorities to increase the diversity of experiences. It also applies a temporary pool to enhance learning from new experiences and changes the fashion of experience pool to first-in-useless-out (FIUO) to make better use of old experiences. In addition, combined with curriculum learning (CL), a more reasonable training paradigm is designed for ACER to train UAV agents smoothly. By training in a large-scale dynamic environment constructed based on the parameters of a real UAV, ACER improves the convergence speed by 24.66% and the convergence result by 5.59% compared to the twin delayed deep deterministic policy gradient (TD3) algorithm. The testing experiments carried out in environments with different complexities further demonstrate the strong robustness and generalization ability of the ACER agents.