Real Unmanned Aerial Vehicles Research Articles

Distributed Multi-Agent Target Tracking: A Nash-Combined Adaptive Differential Evolution Method for UAV Systems

Multi-agent cooperative target tracking plays a significant role in distributed artificial intelligence for unmanned aerial vehicle (UAV) systems. Most existing tracking solutions using online path planning focus on general target trajectories. Such trajectories have strong regularity and are smooth with minimal fluctuations. However, in real UAV applications a target will change its trajectory frequently to prevent from being tracked, and thus its trajectory will fluctuate greatly and randomly. To this end, in this paper we consider a realistic scenario where the UAV target trajectory is changeable and random. For real-time target tracking, we formulate the tracking task as a distributed model predictive control (DMPC) problem, the objective of which is to optimize tracking performance under various constraints. To do so, we innovatively combine the adaptive differential evolution (ADE) algorithm with Nash optimization, and then propose a Nash-combined ADE method. Specifically, we use ADE to adaptively adjust the predicted trajectory of each UAV agent and Nash optimization to efficiently solve DMPC formulation for global optimization, which can improve the tracking accuracy and simultaneously reduce the computational complexity. The simulation results show that the proposed method performs well in terms of tracking accuracy, UAV collision avoidance and tracking stability for realistic target trajectories.

Unmanned Aerial Vehicle Mid-Air Collision Detection and Resolution Using Avoidance Maps

Collision avoidance for fixed-wing unmanned aerial vehicles (UAVs), using a novel methodology of avoidance map, is addressed in this work. Avoidance map is defined as a mapping in the control input space of a pair of UAVs, which partitions the control space into avoidance and collision regions, respectively. The choice of maneuvers to execute successful avoidance can be effectively obtained by using the avoidance map. The duration of control application is determined by a simple method that is time optimal and computationally efficient. Different versions of the map portraying the evolution of the concept are also given. Precision control, defined as a gradual reduction in control, is discussed to reduce the control input demand for the entire duration of avoidance. The concept is extended to multiple UAVs, and the additional requirement of the UAVs to return to nominal trajectory after avoidance, using Dubins paths, is also addressed. Finally, simulation results using realistic UAV models are considered, and some needed modifications to the basic avoidance strategy, made necessary by realistic maneuver models, are discussed. Several illustrative simulation results are given to support the applicability of the proposed method.

Improved Q-Learning Algorithm Based on Approximate State Matching in Agricultural Plant Protection Environment.

An Unmanned Aerial Vehicle (UAV) can greatly reduce manpower in the agricultural plant protection such as watering, sowing, and pesticide spraying. It is essential to develop a Decision-making Support System (DSS) for UAVs to help them choose the correct action in states according to the policy. In an unknown environment, the method of formulating rules for UAVs to help them choose actions is not applicable, and it is a feasible solution to obtain the optimal policy through reinforcement learning. However, experiments show that the existing reinforcement learning algorithms cannot get the optimal policy for a UAV in the agricultural plant protection environment. In this work we propose an improved Q-learning algorithm based on similar state matching, and we prove theoretically that there has a greater probability for UAV choosing the optimal action according to the policy learned by the algorithm we proposed than the classic Q-learning algorithm in the agricultural plant protection environment. This proposed algorithm is implemented and tested on datasets that are evenly distributed based on real UAV parameters and real farm information. The performance evaluation of the algorithm is discussed in detail. Experimental results show that the algorithm we proposed can efficiently learn the optimal policy for UAVs in the agricultural plant protection environment.

BL-ALM: A Blind Scalable Edge-Guided Reconstruction Filter for Smart Environmental Monitoring Through Green IoMT-UAV Networks

Today, low-cost radar/optical visual sensing for monitoring of environment and remote sensing through a network of unmanned aerial vehicles (UAVs) has received more attention such that many different intelligent and pervasive computing techniques are being used to make vision-assisted UAV Networks more reliable and dependable in order to distributed surveillance. On the other hand, the issue of visual information reconstruction has been an interesting topic of machine vision and UAV-borne remote sensing in recent years. To do this, a non-linear blind edge-guided spatial filter based on linear minimum mean square error-estimation (LMMSE) theory has recently been proposed for video sequence reconstruction problems in green monitoring with radars. Although performance of this technique is acceptable compared to the conventional linear techniques in the area, the strategy behind this visual estimator is not generally flexible on which it is not applicable in scalable zooming and its corresponding reconstruction templates. The main objective of this research is to introduce a new version of this filter towards scalable zooming and flexibility in different templates with arbitrary scales. Thus, the proposed algorithm can simultaneously benefit from both edge-directed and distance-weighted adaptation. The proposed approach is theoretically proven, and with using experiments through real UAV visual data, its suitability for low-cost sensing in green Internet of multimedia things (G-IoMT) is shown.

Embedded real-time infrared and visible image fusion for UAV surveillance

Infrared and visible image fusion is a beneficial processing task for Unmanned Aerial Vehicle (UAV) surveillance, which can improve visibility by combining the advantages of the infrared camera and the visible light camera. An embedded onboard solution is necessary for UAV-based surveillance missions because it reduces the amount of data that are transmitted to the ground. In this paper, we propose an infrared and visible light image fusion method and implement it on two platforms with commonly used HW accelerators for embedded vision applications: Zedboard (ARM + FPGA) and NVIDIA TX1 (ARM + GPU), and compare their performances. To verify the usefulness of image fusion, we carry out sufficient experiments to prove that image fusion can improve the target detection ability of a UAV in different scenes. The detection rate for target detection is up to 0.926 in our experiments. The execution times on the ZedBoard and the TX1 are, respectively, 205.3 FPS and 36.6 FPS (38 $$\times$$ and 6.7 $$\times$$ in comparison to an ARM Cortex-A9 processor). Our results also show that the ZedBoard achieves an energy/frame reduction ratio of 7.1 $$\times$$ and 18.9 $$\times$$ respectively compared to the TX1 and the ARM CPU. This work is based on a UAV platform designed by ourselves, and all image sets are real scenes that we have captured. This demonstrates that the proposed method is viable and reflects the actual needs of real UAV surveillance systems.

A novel resilient and reconfigurable swarm management scheme

As we witness the fast growth of the Unmanned Aerial Vehicles (UAVs) field, new applications and services emerge at a rapid pace. Above all, the interest in groups of UAVs working together (swarms) is gaining momentum. This interest emerges since swarms are able to undertake more sophisticated tasks. Furthermore, they can also increase task performance and/or robustness. However, organizing a multi-UAV flight is not easy, involving challenges in terms of (i) swarm formation definition, (ii) takeoff procedure, (iii) in-flight coordination, (iv) communication between the swarm elements, (v) swarm layout reconfiguration, (vi) handling the loss of swarms elements, and (vii) controlled landing. These and other issues still hold back the mainstream adoption of swarms in sectors such as agriculture, border surveillance, and parcel delivery.In this work we provide solutions for two of the main critical challenges: (a) swarm layout reconfiguration, and (b) handling the loss of swarm elements. A wide set of experiments were made using our own realistic UAV emulation tool (ArduSim) in order to validate our proposals. The experiments show that the chances of facing collisions during the reconfiguration are greatly reduced even in error-prone scenarios, and that, in many cases, the loss of a UAV is handled seamlessly; otherwise (in the worst-case scenarios) a delay of just a few seconds is introduced. Additionally, this work addresses cases where, due to the lack of proper communication between the swarm elements, a swarm splits up. Experiments show that with our swarm resilience mechanisms those cases are inherently solved by creating autonomous sub-swarms which will then complete their part of the mission independently.

The CIPCA-BPNN Failure Prediction Method Based on Interval Data Compression and Dimension Reduction

This paper proposes a complete-information-based principal component analysis (CIPCA)-back-propagation neural network (BPNN)_ fault prediction method using real unmanned aerial vehicle (UAV) flight data. Unmanned aerial vehicles are widely used in commercial and industrial fields. With the development of UAV technology, it is imperative to diagnose and predict UAV faults and improve their safety and reliability. The data-driven fault prediction method provides a basis for UAV fault prediction. A UAV is a typical complex system. Its flight data is a kind of typical high-dimensional large sample dataset, and traditional methods cannot meet the requirements of data compression and dimensionality reduction at the same time. The method used interval data to compress UAV flight data, used CIPCA to reduce the dimensionality of the compressed data, and then used a back propagation (BP) neural network to predict UAV failure. Experimental results show that the CIPCA-BPNN method had obvious advantages over the traditional principal component analysis (PCA)-BPNN method and could accurately predict a failure about 9 s before the UAV failure occurred.

Real time UAV path planning by parallel grey wolf optimization with align coefficient on CAN bus

Unmanned aerial vehicle (UAV) path planning is a complex optimization problem, which aims to achieve an optimal or nearly optimal flight path despite various threats and constraints. In this paper, an improved version of Gray Wolf Optimization (GWO) is proposed to solve the UAV 3D path planning problem which considers the dynamics of the UAV. In improved GWO, a variable weighting called "align coefficient" is defined to deal with the problem of waypoint scattering. The parallel GWO is applied to reduce the computation time which makes the possibility of real-time implementation. Given the existence and unique features of CAN bus in UAVs, it is used as a platform to migrate individuals in the parallelization process. The simulation results demonstrate that applying improved GWO generates better performance for UAV 3D path planning problems compared to the conventional GWO, GA, PSO, SA, improved GA and improved PSO.

UAV Localization Using Autoencoded Satellite Images

We propose and demonstrate a fast, robust method for using satellite images to localize an Unmanned Aerial Vehicle (UAV). Previous work using satellite images has large storage and computation costs and is unable to run in real time. In this work, we collect Google Earth (GE) images for a desired flight path offline and an autoencoder is trained to compress these images to a low-dimensional vector representation while retaining the key features. This trained autoencoder is used to compress a real UAV image, which is then compared to the precollected, nearby, autoencoded GE images using an inner-product kernel. This results in a distribution of weights over the corresponding GE image poses and is used to generate a single localization and associated covariance to represent uncertainty. Our localization is computed in 1% of the time of the current standard and is able to achieve a comparable RMSE of less than 3 m in our experiments, where we robustly matched UAV images from six runs spanning the lighting conditions of a single day to the same map of satellite images.

Safety-enhanced UAV path planning with spherical vector-based particle swarm optimization

This paper presents a new algorithm named spherical vector-based particle swarm optimization (SPSO) to deal with the problem of path planning for unmanned aerial vehicles (UAVs) in complicated environments subjected to multiple threats. A cost function is first formulated to convert the path planning into an optimization problem that incorporates requirements and constraints for the feasible and safe operation of the UAV. SPSO is then used to find the optimal path that minimizes the cost function by efficiently searching the configuration space of the UAV via the correspondence between the particle position and the speed, turn angle and climb/dive angle of the UAV. To evaluate the performance of SPSO, eight benchmarking scenarios have been generated from real digital elevation model maps. The results show that the proposed SPSO outperforms not only other particle swarm optimization (PSO) variants including the classic PSO, phase angle-encoded PSO and quantum-behave PSO but also other state-of-the-art metaheuristic optimization algorithms including the genetic algorithm (GA), artificial bee colony (ABC), and differential evolution (DE) in most scenarios. In addition, experiments have been conducted to demonstrate the validity of the generated paths for real UAV operations. Source code of the algorithm can be found at https://github.com/duongpm/SPSO.

Drone Secure Communication Protocol for Future Sensitive Applications in Military Zone.

Unmanned Aerial Vehicle (UAV) plays a paramount role in various fields, such as military, aerospace, reconnaissance, agriculture, and many more. The development and implementation of these devices have become vital in terms of usability and reachability. Unfortunately, as they become widespread and their demand grows, they are becoming more and more vulnerable to several security attacks, including, but not limited to, jamming, information leakage, and spoofing. In order to cope with such attacks and security threats, a proper design of robust security protocols is indispensable. Although several pieces of research have been carried out with this regard, there are still research gaps, particularly concerning UAV-to-UAV secure communication, support for perfect forward secrecy, and provision of non-repudiation. Especially in a military scenario, it is essential to solve these gaps. In this paper, we studied the security prerequisites of the UAV communication protocol, specifically in the military setting. More importantly, a security protocol (with two sub-protocols), that serves in securing the communication between UAVs, and between a UAV and a Ground Control Station, is proposed. This protocol, apart from the common security requirements, achieves perfect forward secrecy and non-repudiation, which are essential to a secure military communication. The proposed protocol is formally and thoroughly verified by using the BAN-logic (Burrow-Abadi-Needham logic) and Scyther tool, followed by performance evaluation and implementation of the protocol on a real UAV. From the security and performance evaluation, it is indicated that the proposed protocol is superior compared to other related protocols while meeting confidentiality, integrity, mutual authentication, non-repudiation, perfect forward secrecy, perfect backward secrecy, response to DoS (Denial of Service) attacks, man-in-the-middle protection, and D2D (Drone-to-Drone) security.

5G-Enabled UAVs with Command and Control Software Component at the Edge for Supporting Energy Efficient Opportunistic Networks

Recently Unmanned Aerial Vehicles (UAVs) have evolved considerably towards real world applications, going beyond entertaining activities and use. With the advent of Fifth Generation (5G) cellular networks and the number of UAVs to be increased significantly, it is created the opportunity for UAVs to participate in the realisation of 5G opportunistic networks by carrying 5G Base-Stations to under-served areas, allowing the provision of bandwidth demanding services, such as Ultra High Definition (UHD) video streaming, as well as other multimedia services. Among the various improvements that will drive this evolution of UAVs, energy efficiency is considered of primary importance since will prolong the flight time and will extend the mission territory. Although this problem has been studied in the literature as an offline resource optimisation problem, the diverse conditions of a real UAV flight does not allow any of the existing offline optimisation models to be applied in real flight conditions. To this end, this paper discusses the amalgamation of UAVs and 5G cellular networks as an auspicious solution for realising energy efficiency of UAVs by offloading at the edge of the network the Flight Control System (FCS), which will allow the optimisation of the UAV energy resources by processing in real time the flight data that have been collected by onboard sensors. By exploiting the Multi-access Edge Computing (MEC) architectural feature of 5G as a technology enabler for realising this offloading, the paper presents a proof-of-concept implementation of such a 5G-enabled UAV with softwarized FCS component at the edge of the 5G network (i.e., the MEC), allowing by this way the autonomous flight of the UAV over the 5G network by following control commands mandated by the FCS that has been deployed at the MEC. This proof-of-concept 5G-enabled UAV can support the execution of real-time resource optimisation techniques, a step-forward from the currently offline-ones, enabling in the future the execution of energy-efficient and advanced missions.

Mathematical Modeling of the Coaxial Quadrotor Dynamics for Its Attitude and Altitude Control

In this paper, an easily implementable coaxial quadrotor model and its validation on data from a real unmanned aerial vehicle (UAV), are presented. The proposed mathematical model consists of two parts: description of orientation and position of the UAV in the three-dimensional space. It takes into consideration the gyroscopic effect, influence of the Coriolis force, viscous friction and a several drag-like effects (blade flapping, rotor drag, translational drag and profile drag). In contrast to multirotor models available in the literature, this one is characterized by complementarity in relation to the available control techniques. Depending on selection of these techniques, the model can be narrowed (simplified) to meet the needs without the loss of behaviour adequacy to a real UAV.

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles (UAVs) play a vital role in military warfare. In a variety of battlefield mission scenarios, UAVs are required to safely fly to designated locations without human intervention. Therefore, finding a suitable method to solve the UAV Autonomous Motion Planning (AMP) problem can improve the success rate of UAV missions to a certain extent. In recent years, many studies have used Deep Reinforcement Learning (DRL) methods to address the AMP problem and have achieved good results. From the perspective of sampling, this paper designs a sampling method with double-screening, combines it with the Deep Deterministic Policy Gradient (DDPG) algorithm, and proposes the Relevant Experience Learning-DDPG (REL-DDPG) algorithm. The REL-DDPG algorithm uses a Prioritized Experience Replay (PER) mechanism to break the correlation of continuous experiences in the experience pool, finds the experiences most similar to the current state to learn according to the theory in human education, and expands the influence of the learning process on action selection at the current state. All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV. The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm, while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.

Assisting UAV Localization Via Deep Contextual Image Matching

In this article, we aim to explore the potential of using onboard cameras and pre-stored geo-referenced imagery for Unmanned Aerial Vehicle (UAV) localization. Such a vision-based localization enhancing system is of vital importance, particularly in situations where the integrity of the global positioning system (GPS) is in question (i.e., in the occurrence of GPS outages, jamming, etc.). To this end, we propose a complete trainable pipeline to localize an aerial image in a pre-stored orthomosaic map in the context of UAV localization. The proposed deep architecture extracts the features from the aerial imagery and localizes it in a pre-ordained, larger, and geotagged image. The idea is to train a deep learning model to find neighborhood consensus patterns that encapsulate the local patterns in the neighborhood of the established dense feature correspondences by introducing semi-local constraints. We qualitatively and quantitatively evaluate the performance of our approach on real UAV imagery. The training and testing data is acquired via multiple flights over different regions. The source code along with the entire dataset, including the annotations of the collected images has been made public. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sup> https://github.com/m-hamza-mughal/Aerial-Template-Matching. Up-to our knowledge, such a dataset is novel and first of its kind which consists of 2052 high-resolution aerial images acquired at different times over three different areas in Pakistan spanning a total area of around 2 km <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> .

Coupling of machine learning methods to improve estimation of ground coverage from unmanned aerial vehicle (UAV) imagery for high-throughput phenotyping of crops.

Ground coverage (GC) allows monitoring of crop growth and development and is normally estimated as the ratio of vegetation to the total pixels from nadir images captured by visible-spectrum (RGB) cameras. The accuracy of estimated GC can be significantly impacted by the effect of 'mixed pixels', which is related to the spatial resolution of the imagery as determined by flight altitude, camera resolution and crop characteristics (fine vs coarse textures). In this study, a two-step machine learning method was developed to improve the accuracy of GC of wheat (Triticum aestivum L.) estimated from coarse-resolution RGB images captured by an unmanned aerial vehicle (UAV) at higher altitudes. The classification tree-based per-pixel segmentation (PPS) method was first used to segment fine-resolution reference images into vegetation and background pixels. The reference and their segmented images were degraded to the target coarse spatial resolution. These degraded images were then used to generate a training dataset for a regression tree-based model to establish the sub-pixel classification (SPC) method. The newly proposed method (i.e. PPS-SPC) was evaluated with six synthetic and four real UAV image sets (SISs and RISs, respectively) with different spatial resolutions. Overall, the results demonstrated that the PPS-SPC method obtained higher accuracy of GC in both SISs and RISs comparing to PPS method, with root mean squared errors (RMSE) of less than 6% and relative RMSE (RRMSE) of less than 11% for SISs, and RMSE of less than 5% and RRMSE of less than 35% for RISs. The proposed PPS-SPC method can be potentially applied in plant breeding and precision agriculture to balance accuracy requirement and UAV flight height in the limited battery life and operation time.

Hybrid deep learning‐based throughput analysis for UAV‐assisted cellular networks

Mobile users are interested in utilising high network capabilities without time and place constraints. However, with a high level of interest in the usage of mobile phones and internet facilities, the limited capacity of terrestrial base stations (BSs) is unbalanced. As a potential alternative to BSs, unmanned aerial vehicles (UAVs) are emerging as a means of transmitting wireless data to ground mobile users. As an air-to-ground communication network, the real UAVs deployed and collected communication data from ground mobile users. The main objective of this study is to analyse and evaluate user throughput, interference, and power transmission when the UAVs are at different heights. The parameters used include the locations of the UAVs and users, the altitudes and elevation angles from the users to UAVs, signal-to-noise-ratio, throughput values, the categories of line-of-sight, and non-line-of-sight links. Furthermore, K-means used as a clustering method for class identification, long short-term memory (LSTM), and gated recurrent unit (GRU) to analyse and evaluate system performance. The system's performance was compared with a multi-layer perceptron approach. The evaluation results show that the proposed LSTM–GRU provides reliable and encouraging performance with low computational complexity, which is appropriate for heterogeneous networks.

Numerical and experimental verification of an inverse‐direct approach for load and strain monitoring in aeronautical structures

Aeronautical structures are increasingly aging, and the occurrence of unexpected loads could reduce their operability. A health and usage monitoring system would enable the continuous monitoring of the state of health of a structure and track its aging by a load monitoring system, which aims at the real-time reconstruction of the loads acting on a structure. However, sometimes the loads and the induced strain and stress fields are difficult to be reconstructed exactly, as for complex loading due to flight maneuvers. In this work, the full strain and load fields of a structure are reconstructed by an inverse-direct approach, leveraging on the calibration matrix approach. The latter exploits a least-squares minimization of an error functional, defined as the comparison between measured strains in discrete positions and a numerical formulation of the same, to reconstruct an equivalent, however representative, load set. By assuming a linear relationship between strain and load through a calibration matrix, this minimization can be performed analytically, leading to a computationally very efficient algorithm that can be operated online. Once the equivalent load set is computed, the full strain field can be estimated relying on a second calibration matrix linking the external loads to the strain field of the complete structure. The method has been numerically tested with an unmanned aerial vehicle (UAV) subjected to aerodynamic pressure loads simulating flight maneuvers. Finally, the results are experimentally validated during a ground test program on a real UAV, proving the robustness to different experimental uncertainties.

Validation of an IMU-camera fusion algorithm using an industrial robot

The integration of down-looking camera with an in-ertial measurement unit (IMU) sensor makes possible to provide a lightweight and low-cost pose estimation system for unmanned aerial vehicles (UAVs) and micro-UAVs (MAVs). Recently, the authors developed an algorithm for IMU and exteroceptive sensor fusion filter for position and orientation estimation. The aim of the estimation is to be used in the outer control loop of an UAV for position control. This work presents an experimental set up to test that algorithm using an industrial robot to produce accurate planar trajectories as a safe alternative to testing the algorithm on real UAVs. The results of the IMU-camera fusion estimation for linear positions and linear velocities show an error admissible to be integrated on real UAVs.

Adaptive autonomous UAV scouting for rice lodging assessment using edge computing with deep learning EDANet

Rice is a globally important crop that will continue to play an essential role in feeding our world as we grapple with climate change and population growth. Lodging is a primary threat to rice production, decreasing rice yield, and quality. Lodging assessment is a tedious task and requires heavy labor and a long duration due to the vast land areas involved. Newly developed autonomous crop scouting techniques have shown promise in mapping crop fields without any human interaction. By combining autonomous scouting and lodged rice detection with edge computing, it is possible to estimate rice lodging faster and at a much lower cost than previous methods. This study presents an adaptive crop scouting mechanism for Autonomous Unmanned Aerial Vehicles (UAV). We simulate UAV crop scouting of rice fields at multiple levels using deep neural networks and real UAV energy profiles, focusing on areas with high lodging. Using the proposed method, we can scout rice fields 36% faster than conventional scouting methods at 99.25% accuracy.