Multiple Unmanned Aerial Vehicles Research Articles

Decentralized Adaptive Event-Triggered Fault-Tolerant Cooperative Control of Multiple Unmanned Aerial Vehicles and Unmanned Ground Vehicles with Prescribed Performance under Denial-of-Service Attacks

Decentralized Adaptive Event-Triggered Fault-Tolerant Cooperative Control of Multiple Unmanned Aerial Vehicles and Unmanned Ground Vehicles with Prescribed Performance under Denial-of-Service Attacks

Formation control of high-order linear discrete-time multiple unmanned aerial vehicles systems in noisy environments

Formation control of high-order linear discrete-time multiple unmanned aerial vehicles (multi-UAVs) systems with multiplicative noises is studied in this article. Because of the existence of multiplicative noises, classical methods for discrete-time system control problems should be improved. Here, we develop a new discrete-time formation protocol for multi-UAVs systems. The stochastic Lyapunov stability theory and the positive definiteness theory of a generalized algebraic Riccati equation are then used to derive sufficient conditions for formation feasibility.

Multiple Unmanned Aerial Vehicle (multi-UAV) Reconnaissance and Search with Limited Communication Range Using Semantic Episodic Memory in Reinforcement Learning

Unmanned Aerial Vehicles (UAVs) have garnered widespread attention in reconnaissance and search operations due to their low cost and high flexibility. However, when multiple UAVs (multi-UAV) collaborate on these tasks, a limited communication range can restrict their efficiency. This paper investigates the problem of multi-UAV collaborative reconnaissance and search for static targets with a limited communication range (MCRS-LCR). To address communication limitations, we designed a communication and information fusion model based on belief maps and modeled MCRS-LCR as a multi-objective optimization problem. We further reformulated this problem as a decentralized partially observable Markov decision process (Dec-POMDP). We introduced episodic memory into the reinforcement learning framework, proposing the CNN-Semantic Episodic Memory Utilization (CNN-SEMU) algorithm. Specifically, CNN-SEMU uses an encoder–decoder structure with a CNN to learn state embedding patterns influenced by the highest returns. It extracts semantic features from the high-dimensional map state space to construct a smoother memory embedding space, ultimately enhancing reinforcement learning performance by recalling the highest returns of historical states. Extensive simulation experiments demonstrate that in reconnaissance and search tasks of various scales, CNN-SEMU surpasses state-of-the-art multi-agent reinforcement learning methods in episodic rewards, search efficiency, and collision frequency.

A learning-based memetic algorithm for a cooperative task allocation problem of multiple unmanned aerial vehicles in smart agriculture

Smart agriculture aligns with the principles of sustainable development, making it a crucial direction for the future agriculture. This study focuses on a cooperative plant protection task allocation problem (CPPTAP) of multiple unmanned aerial vehicles (UAVs) with a common deadline in smart agriculture. CPPTAP permits multiple UAVs to conduct pesticide spraying on the same field. The completion time for each task fluctuates due to the cooperation among UAVs. We present a mathematical model and learning-based memetic algorithm (L-MA) to maximize the total area of the fields to be sprayed. In the evolutionary stage, mutation and repair operators based on value information are applied to balance the exploration and exploitation, while a problem-specific local search strategy is designed to enhance exploitation capability. A knowledge-based UAV allocation method (KUAM) is employed to maximize UAV utilization efficiency and minimize conflicts. Throughout the search process, Q-learning is utilized to assist the aforementioned operators and make decisions on the number of cooperative UAVs on fields. The effectiveness of L-MA is validated by comparing it against other state-of-the-art algorithms. The results demonstrate that L-MA outperforms the compared algorithms at a considerable margin in a statistical sense.

A Heuristic Routing Algorithm for Heterogeneous UAVs in Time-Constrained MEC Systems

The rapid proliferation of Internet of Things (IoT) ground devices (GDs) has created an unprecedented demand for computing resources and real-time data-processing capabilities. Integrating unmanned aerial vehicles (UAVs) into Mobile Edge Computing (MEC) emerges as a promising solution to bring computation and storage closer to the data sources. However, UAV heterogeneity and the time window constraints for task execution pose a significant challenge. This paper addresses the multiple heterogeneity UAV routing problem in MEC environments, modeling it as a multi-traveling salesman problem (MTSP) with soft time constraints. We propose a two-stage heuristic algorithm, heterogeneous multiple UAV routing (HMUR). The approach first identifies task areas (TAs) and optimal hovering positions for the UAVs and defines an effective fitness measurement to handle UAV heterogeneity. A novel scoring function further refines the path determination, prioritizing real-time task compliance to enhance Quality of Service (QoS). The simulation results demonstrate that our proposed HMUR method surpasses the existing baseline algorithms on multiple metrics, validating its effectiveness in optimizing resource scheduling in MEC environments.

Multi-Unmanned Aerial Vehicle-Assisted Flood Navigation of Waterborne Vehicles Using Deep Reinforcement Learning

Abstract During disasters, such as floods, it is crucial to get real-time ground information for planning rescue and response operations. With the advent of technology, unmanned aerial vehicles (UAVs) are being deployed for real-time path planning to provide support to evacuation teams. However, their dependency on expert human pilots for command and control limits their operational capacity to the line-of-sight range. In this article, we utilize a deep reinforcement learning algorithm to autonomously control multiple UAVs for area coverage. The objective is to identify serviceable paths for safe navigation of waterborne evacuation vehicles (WBVs) to reach critical location(s) during floods. The UAVs are tasked to capture the obstacle-related data and identify shallow water regions for unrestricted motion of the WBV(s). The data gathered by UAVs is used by the minimum expansion A* (MEA*) algorithm for path planning to assist WBV(s). MEA* addresses the node expansion issue with the standard A* algorithm, by pruning the unserviceable nodes/locations based on the captured information, hence expediting the path planning process. The proposed approach, MEA*MADDPG, is compared with other prevalent techniques from the literature over simulated flood environments with moving obstacles. The results highlight the significance of the proposed model as it outperforms other techniques when compared over various performance metrics.

Learning continuous multi-UAV controls with directed explorations for flood area coverage

Real time on-ground information is of critical value during any natural disaster such as floods. The disaster response teams require the latest ground information of the flooded areas to effectively plan and execute rescue operations. Unmanned Aerial Vehicles (UAVs) are increasingly becoming a tool to perform quick surveys of larger areas such as flood disasters. In this paper, we propose a method to perform critical area coverage of flood-struck regions using multiple autonomous UAVs. A Deep Reinforcement Learning algorithm is proposed to learn continuous multi-UAV controls, incorporating a directed exploration strategy for the DDPG’s target actor, which relies on the D-infinity (DINF) algorithm. The DINF water flow estimation technique utilizes surface elevation data to understand and predict the directed discharge of floodwater. Further, we introduce a Path scatter strategy for the multi-UAV system that inhibits the clustered formation of the UAVs over low-elevated regions. The performance of the proposed D3S (DDPG+DINF+Path Scatter) algorithm is evaluated using various performance metrics, such as average cumulative rewards, number of collisions, and UAVs’ spread observed over the environment. In comparison to the baseline algorithms and other prevalent approaches in the literature, the proposed method is found to be better placed as the results highlight a significantly improved performance by D3S across different metrics.

Leveraging Edge Computing for Video Data Streaming in UAV-Based Emergency Response Systems.

The rapid advancement of technology has greatly expanded the capabilities of unmanned aerial vehicles (UAVs) in wireless communication and edge computing domains. The primary objective of UAVs is the seamless transfer of video data streams to emergency responders. However, live video data streaming is inherently latency dependent, wherein the value of the video frames diminishes with any delay in the stream. This becomes particularly critical during emergencies, where live video streaming provides vital information about the current conditions. Edge computing seeks to address this latency issue in live video streaming by bringing computing resources closer to users. Nonetheless, the mobile nature of UAVs necessitates additional trajectory supervision alongside the management of computation and networking resources. Consequently, efficient system optimization is required to maximize the overall effectiveness of the collaborative system with limited UAV resources. This study explores a scenario where multiple UAVs collaborate with end users and edge servers to establish an emergency response system. The proposed idea takes a comprehensive approach by considering the entire emergency response system from the incident site to video distribution at the user level. It includes an adaptive resource management strategy, leveraging deep reinforcement learning by simultaneously addressing video streaming latency, UAV and user mobility factors, and varied bandwidth resources.

Collision-Avoiding Flocking With Multiple Fixed-Wing UAVs in Obstacle-Cluttered Environments: A Task-Specific Curriculum- Based MADRL Approach.

Multiple unmanned aerial vehicles (UAVs) are able to efficiently accomplish a variety of tasks in complex scenarios. However, developing a collision-avoiding flocking policy for multiple fixed-wing UAVs is still challenging, especially in obstacle-cluttered environments. In this article, we propose a novel curriculum-based multiagent deep reinforcement learning (MADRL) approach called task-specific curriculum-based MADRL (TSCAL) to learn the decentralized flocking with obstacle avoidance policy for multiple fixed-wing UAVs. The core idea is to decompose the collision-avoiding flocking task into multiple subtasks and progressively increase the number of subtasks to be solved in a staged manner. Meanwhile, TSCAL iteratively alternates between the procedures of online learning and offline transfer. For online learning, we propose a hierarchical recurrent attention multiagent actor-critic (HRAMA) algorithm to learn the policies for the corresponding subtask(s) in each learning stage. For offline transfer, we develop two transfer mechanisms, i.e., model reload and buffer reuse, to transfer knowledge between two neighboring stages. A series of numerical simulations demonstrate the significant advantages of TSCAL in terms of policy optimality, sample efficiency, and learning stability. Finally, the high-fidelity hardware-in-the-loop (HITL) simulation is conducted to verify the adaptability of TSCAL. A video about the numerical and HITL simulations is available at https://youtu.be/R9yLJNYRIqY.

Graph-based multi-agent reinforcement learning for collaborative search and tracking of multiple UAVs

This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.

Improved PSO-Based Two-Phase Logistics UAV Path Planning under Dynamic Demand and Wind Conditions

Unmanned aerial vehicles (UAVs) have increasingly become integral to logistics and distribution due to their flexibility and mobility. However, the existing studies often overlook the dynamic nature of customer demands and wind conditions, limiting the practical applicability of their proposed strategies. To tackle this challenge, we firstly construct a time-slicing-based UAV path planning model that incorporates dynamic customer demands and wind impacts. Based on this model, a two-stage logistics UAV path planning framework is developed according to the analysis of the customer pool updates and dynamic attitudes. Secondly, a dynamic demand and wind-aware logistics UAV path planning problem is formulated to minimize the weighted average of the energy consumption and the customer satisfaction penalty cost, which comprehensively takes the energy consumption constraints, load weight constraints, and hybrid time window constraints into consideration. To solve this problem, an improved particle swarm optimization (PSO)-based multiple logistics UAV path planning algorithm is developed, which has good performance with fast convergence and better solutions. Finally, extensive simulation results verify that the proposed algorithm can not only adhere to the UAV’s maximum load and battery power constraints but also significantly enhance the loading efficiency and battery utilization rate. Particularly, compared to the genetic algorithm (GA), simulated annealing (SA), and traditional PSO strategies, our proposed algorithm achieves satisfactory solutions within a reasonable time frame and reduces the distribution costs by up to 9.82%.

Two-stage knowledge-assisted coevolutionary NSGA-II for bi-objective path planning of multiple unmanned aerial vehicles

This paper focuses on the bi-objective path planning problem of multiple unmanned aerial vehicles (UAVs) under the complex environment with numerous obstacles and threat areas, where the UAVs need to be kept as far away as possible from threat areas during flight. Based on the integrated energy reduction perspective, a bi-objective model is subtly constructed by minimizing the total energy consumption of each path (including flight altitude, horizontal turns, and path length), and minimizing the costs of the total threats (including ground radar, anti-aircraft gun, missile and geological hazard threat areas). Moreover, a two-stage knowledge-assisted coevolutionary NSGA-II algorithm is novelly proposed to enhance collaboration and avoid collision. The first stage is designed for population convergence, where the considered constrained problem is solved with the help of the designed problem without the constraints of threats and obstacles. The second stage emphasizes the quality and diversity of solutions. In this stage, a double-population coevolution approach is developed. Additionally, a multi-mode strategy is introduced for the inferior population, leveraging reinforcement learning. This strategy aids in selecting the optimal mode from random swing, directed guidance, and potential dominance exploration. Furthermore, experimental results in two different environments show that the proposed algorithm can better solve the collaborative path planning problem for multiple UAVs compared with other five classical or recent proposed algorithms.

Research on collaborative multi-UAV localization method based on combination navigation information

In challenging environments, unmanned aerial vehicle (UAV) systems often encounter unstable satellite signals and communication link interference. This paper proposes an integrated navigation method that integrates inertial navigation system (INS), global navigation satellite system (GNSS), and visual navigation system (VNS). Utilizing data from onboard sensors, this method merges relative navigation information from feature tracking of multiple UAVs with each UAV’s absolute navigation data. It includes specially designed transmission rules to reduce data exchange between UAVs. Each UAV uses an adaptive unscented Kalman filter (AUKF) method, which is enhanced into a collaborative AUKF (C-AUKF) using a message passing-based approach. Experiments in a simulated mission scenario revealed that the C-AUKF, in comparison to using extended Kalman filter (EKF), significantly improved flight test performance across the entire testing area, with a cumulative deviation of only 10.22 m, about 0.85% of the total flight distance. These results demonstrate that the proposed method not only meets accuracy requirements for position and velocity in integrated navigation but also significantly enhances multi-UAV navigation precision, particularly in scenarios with global positioning system (GPS) interference.

MURE: Multi-layer real-time livestock management architecture with unmanned aerial vehicles using deep reinforcement learning

In recent years, the combination of unmanned aerial vehicles (UAVs) and wireless sensor networks (WSNs) has gained popularity in livestock management (LM) due to energy constraints and network instability. Limited energy storage of sensor nodes (SNs) and the possibility of packet loss contribute to fast energy consumption and unstable networks, respectively. UAVs serve as relay nodes and data sinks, addressing these issues by temporarily storing data to reduce SN workload and establishing mobile nodes for network stability. We propose two innovations based on previous work: 1) We introduce a multi-layer wireless network architecture, categorizing UAVs into two layers based on their functions including data collection and data processing. This enhances task parallelization, bridging performance gaps among multiple UAVs; 2) We overcome the mobility limitation of SNs, considering their real-time movement in the network. Through deep reinforcement learning, UAVs learn to cooperatively locate moving SNs. This accounts for the inevitable mobility of livestock in the industry. Additionally, we simulate the environment and compare our approach to traditional methods, evaluating metrics such as collected data per timestep (DCPS), energy consumed per timestep (ECPS), and network stability (NS). Experimental results demonstrate that our method outperforms traditional approaches, achieving a data collecting gain of 4.84% and 8.20% compared to the methods without considering SN mobility or the multi-layer characteristics of WSNs, respectively. Under energy consumption limits, our method yields energy savings of 3.00% and 1.35% respectively. Furthermore, we extensively study and validate our method against other path planning algorithms, including genetic particle swarm optimization (GPSO), modified central force optimization (MCFO), and rapidly-exploring random trees (RRT). Our approach surpasses these methods in terms of data collecting efficiency and network stability.

Multiple UAVs Networking Oriented Consistent Cooperation Method Based on Adaptive Arithmetic Sine Cosine Optimization

With the rapid development of the Internet of Things, the Internet of Vehicles (IoV) has quickly drawn considerable attention from the public. The cooperative unmanned aerial vehicles (UAVs)-assisted vehicular networks, as a part of IoV, has become an emerging research spot. Due to the significant limitations of the application and service of a single UAV-assisted vehicular networks, efforts have been put into studying the use of multiple UAVs to assist effective vehicular networks. However, simply increasing the number of UAVs can lead to difficulties in information exchange and collisions caused by external interference, thereby affecting the security of the entire cooperation and networking. To address the above problems, multiple UAV cooperative formation is increasingly receiving attention. UAV cooperative formation can not only save energy loss but also achieve synchronous cooperative motion through information communication between UAVs, prevent collisions and other problems between UAVs, and improve task execution efficiency. A multi-UAVs cooperation method based on arithmetic optimization is proposed in this work. Firstly, a complete mechanical model of unmanned maneuvering was obtained by combining acceleration limitations. Secondly, based on the arithmetic sine and cosine optimization algorithm, the mathematical optimizer was used to accelerate the function transfer. Sine and cosine strategies were introduced to achieve a global search and enhance local optimization capabilities. Finally, in obtaining the precise position and direction of multi-UAVs to assist networking, the cooperation method was formed by designing the reference controller through the consistency algorithm. Experimental studies were carried out for the multi-UAVs’ cooperation with the particle model, combined with the quadratic programming problem-solving technique. The results show that the proposed quadrotor dynamic model provides basic data for cooperation position adjusting, and our simplification in the model can reduce the amount of calculations for the feedback and the parameter changes during the cooperation. Moreover, combined with a reference controller, the UAVs achieve the predetermined cooperation by offering improved navigation speed, task execution efficiency, and cooperation accuracy. Our proposed multi-UAVs cooperation method can improve the quality of service significantly on the UAV-assisted vehicular networks.

Dynamic event-triggered time-varying formation control for heterogeneous unmanned swarm systems with scaling attacks

This paper addresses the problem of time-varying formation control for heterogeneous unmanned swarm systems (USSs) consisting of multiple unmanned aerial vehicles (UASs) and unmanned surface vehicles (USVs) based on a directional switching topology. Firstly, a novel USSs model is established, involving the effect of multiple network attacks, network bandwidth constraints and input saturation. In particular, a scaling attack model is proposed to account for denial-of-service (DoS) and deception attacks. Secondly, a distributed controller based on a dynamic event triggering mechanism is designed to reduce the communication load by adjusting the communication frequency between individuals, and the Zeno phenomenon is effectively avoided. Finally, the Lyapunov function is used to obtain sufficient conditions for system stability and bounds on the frequency and duration of network attacks. The experiments and comparison results verify the proposed strategy's effectiveness and superiority.

Connectivity preservation control for multiple unmanned aerial vehicles in the presence of bounded actuation

This paper proposes a novel multi-unmanned aerial vehicle (UAV) connectivity preservation controller, suitable for scenarios with bounded actuation and limited communication range. According to the hierarchical control strategy, controllers are designed separately for the position and attitude subsystems. A distributed position controller is developed, integrating an indirect coupling control mechanism. The innovative mechanism associates each UAV with a virtual proxy, facilitating connections among adjacent UAVs through these proxies. This structuring assists in managing the actuator saturation constraints effectively. The artificial potential function is utilized to preserve network connectivity and fulfill coordination among all virtual proxies. Additionally, an attitude controller designed for finite-time convergence guarantees that the attitude subsystem adheres precisely to the attitude specified by the distributed position controller. Simulation results validate the efficacy of this distributed formation controller with connectivity preservation under bounded actuation conditions. The simulation results confirm the effectiveness of the distributed connectivity preservation controller with bounded actuation.

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

Computation offloading, resource allocation, and endurance issues in unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks have always been a research focus. UAV-aided MEC allows mobile users’ (MUs’) tasks to be offloaded to drones for processing in special scenarios, such as natural disasters or military attacks. However, as the number and size of offloaded tasks continuously increase, it is difficult for a single UAV to meet all computational demands, result in the decline of QoS. In order to address this issue, this paper presents a collaborative computation offloading scheme where multiple UAVs can cooperate to handle massive tasks. Firstly, considering that battery-limited UAVs cannot complete all tasks and sustain flight without recharging, we incorporate charging stations (CS) into multi-UAV-assisted MEC networks. Subsequently, we design a price-based incentive mechanism to encourage the adjacent UAVs with unused computation resources to cooperate with busy UAVs, so as to maximize the total revenue obtained from UAVs’ collaborative computation. Then, we formulate the joint optimization problem of computation offloading, resource allocation and charging scheduling as a Markov Decision Process (MDP), and propose a Twin Delayed Deep Deterministic policy gradient (TD3) algorithm to find optimal strategies. Finally, extensive simulations demonstrate that the proposed TD3 algorithm outperforms other benchmark methods, achieving the maximum overall system utility under different scenarios.

Multi-UAV Cooperative Coverage Search for Various Regions Based on Differential Evolution Algorithm.

In recent years, remotely controlling an unmanned aerial vehicle (UAV) to perform coverage search missions has become increasingly popular due to the advantages of the UAV, such as small size, high maneuverability, and low cost. However, due to the distance limitations of the remote control and endurance of a UAV, a single UAV cannot effectively perform a search mission in various and complex regions. Thus, using a group of UAVs to deal with coverage search missions has become a research hotspot in the last decade. In this paper, a differential evolution (DE)-based multi-UAV cooperative coverage algorithm is proposed to deal with the coverage tasks in different regions. In the proposed algorithm, named DECSMU, the entire coverage process is divided into many coverage stages. Before each coverage stage, every UAV automatically plans its flight path based on DE. To obtain a promising flight trajectory for a UAV, a dynamic reward function is designed to evaluate the quality of the planned path in terms of the coverage rate and the energy consumption of the UAV. In each coverage stage, an information interaction between different UAVs is carried out through a communication network, and a distributed model predictive control is used to realize the collaborative coverage of multiple UAVs. The experimental results show that the strategy can achieve high coverage and a low energy consumption index under the constraints of collision avoidance. The favorable performance in DECSMU on different regions also demonstrate that it has outstanding stability and generality.

Time-dependent hydrogen fuel cell vehicle routing problem with drones and variable drone speeds

This paper proposes a novel problem called the time-dependent hydrogen fuel cell vehicle routing problem with drones and variable drone speeds (TDHFCVRP-D-VDS). In this problem, a fleet of hydrogen fuel cell vehicles (HFCVs) are equipped with multiple unmanned aerial vehicles (UAVs) to perform delivery and pickup services within the customers’ time windows. The UAV is capable of performing pickup operations after the delivery service, as long as it does not exceed its energy capacity. The HFCV can launch and retrieve the UAV multiple times as needed throughout the routing process. We establish a mixed integer programming model to simultaneously minimize the total cost and makespan, and verify its accuracy by Gurobi. To tackle larger-scale instances, we propose a non-dominated sorting genetic algorithm III with intelligent selection (NSGA-III-IS). The initial population is generated using four distinct approaches aimed at enhancing both diversity and quality. Considering the customers’ time windows, we factor in the temporal-spatial distances between customers when generating the initial population. Our approach employs a two-phase algorithm for developing initial solutions. In the first phase, the algorithm focuses on generating UAV trips, while in the second phase, it creates joint delivery routes for both the HFCV and the UAV. To enhance the optimization process, we have developed four strategies for optimizing UAV speed. These strategies dynamically adjust the UAV’s speed in response to the customers’ time windows and the HFCVs’ arrival times. Additionally, we have integrated an intelligent selection mechanism to optimize the execution probabilities of both general and problem-specific operators. The experimental results demonstrate the following: (1) The proposed NSGA-III-IS outperforms other variant algorithms and two benchmark algorithms; (2) UAVs significantly benefit from variable flight speeds, resulting in reduced costs and improved efficiency; (3) The HFCV with UAV joint delivery pattern is superior for reducing carbon emissions compared to other joint delivery patterns; (4) Longer customer time windows and optimized UAV speed strategies are effective in reducing the total cost, makespan, and total UAV hover waiting time; (5) Finally, a method that combines multi-attribute decision-making with principal component analysis is utilized to aid decision-makers in selecting satisfactory solutions.