Group Of Unmanned Aerial Vehicles Research Articles

An evolutionary multi-agent reinforcement learning algorithm for multi-UAV air combat

In recent years, multi-agent reinforcement learning (MARL) has been widely used as a multi-UAV autonomous countermeasure. However, because training is usually performed in a self-play manner, the existing algorithms may suffer from strategic cycles without progress. Moreover, in complex air combat, the state dimensions change dynamically with the varying number of unmanned aerial vehicles (UAVs), making it difficult for existing MARL algorithms with fixed-input-size network architectures to effectively handle the variations. Therefore, we propose an evolutionary MARL method that can avoid cycles through the evolution of a population of UAV groups. In addition, an attention mechanism with death masking was adopted to handle dynamic changes in the number of UAVs. Experimental results conducted in our constructed multi-UAV air combat environment show that our method outperforms state-of-the-art methods and can help UAVs generate effective maneuver strategies to defeat opponents.

Distributed formation control of multi-UAV systems using relative information

An adaptive distributed control strategy is designed for the formation of multiple unmanned aerial vehicle (UAV) groups by using only relative measurement information. A distributed estimator is first designed to get the estimate of the consensus error. Then, an adaptive distributed controller is proposed to realize the formation tracking with the relative measurements. Based on the position and velocity information, a control barrier function (CBF) algorithm is applied to achieve the UAV formation under safe position constraints. In addition, in order to break up and reorganize multiple UAV groups, the proposed controller is designed based on the time-varying network topology. It is proved that the closed-loop system is uniformly asymptotically stability. Simulation and experiment results indicate that the proposed strategy can achieve formation control of multiple UAVs under safety constraints based on the measurement of relative measurement information.

Network Analysis of Decentralized Fault-Tolerant UAV Swarm Coordination in critical Missions

Unmanned Aerial Vehicles (UAVs) have gained prominence across various sectors for their versatile applications. While their advantages are evident, addressing concerns associated with their deployment is essential to ensure reliability. This study presents an innovative approach for coordinating a group of UAVs in aerial survey missions. The decentralized strategy presented in this article allow UAVs to self-organize into linear formation, optimize their coverage paths, and adapt to agent failures, thereby ensuring efficient and adaptive mission execution. The strategy has been tested and validated on two different platforms: the inter-UAV communication performance is evaluated on NS-3 simulator to measure metrices such as packet delivery ratio, throughput, delay, and routing overhead within the UAV swarms, while mission efficiency and fault tolerance is analyzed on ROS framework, and visualized on Gazebo simulator with real-time parameters. Through experimental results, we show that, after proper tuning of control parameters, the approach succeeds in flock formation with high level of fault tolerance, offering higher efficiency in terms of mission time, transmission delay, packet delivery rate, and control overhead, when compared to the benchmark approaches.

UAV-Enabled Secure Communications via Collaborative Beamforming With Imperfect Eavesdropper Information

Unmanned aerial vehicles (UAVs) are playing a pivotal role in wireless networks due to their high mobility and on-demand deployment advantages. However, the UAV-enabled communications are susceptible to be wiretapped by eavesdroppers due to the strong line-of-sight (LoS) dominated air-ground channel. In this paper, we consider a UAV-enabled secure communication scenario, in which a group of UAVs form a UAV-enabled virtual antenna array (UVAA) to transmit information towards the remote base stations (BSs) via collaborative beamforming (CB), while multiple known and unknown eavesdroppers aiming to wiretap the information. Specifically, a secure communication multi-objective optimization problem (SCMOP) is formulated to achieve the maximization of the worst-case secrecy rate, the minimization of the maximum sidelobe level (SLL) as well as the minimization of the flight energy consumption of UAVs by obtaining optimal locations and excitation current weights concerning the UAVs as well as determining an optimal receiver BS that can achieve superior communication performance. To solve the formulated SCMOP which is demonstrated to be non-convex and NP-hard, an improved multi-objective salp swarm algorithm (IMSSA) with several specific operating factors is proposed. Simulations results demonstrate that the proposed IMSSA can deal with the formulated SCMOP effectively and outperforms other benchmark strategies. Moreover, the multi-hop relay is introduced to verify the reasonability of the UVAA system, and two benchmark schemes of the formulated SCMOP are introduced to demonstrate the necessity of the formulated SCMOP. In addition, the performance of the UVAA system under certain unexpected circumstances is estimated. Finally, experimental implementation is conducted by using a Raspberry Pi and the results demonstrate the practicality of the proposed CB-based secure communication approach in real-world scenarios.

Graph-based multi-agent reinforcement learning for large-scale UAVs swarm system control

In this study, a novel graph-embedding technique based on a graph neural network (GNN) is proposed to identify the topology in the motion of a unmanned aerial vehicles (UAV) swarm and quickly obtain local information around each agent. We also propose a model reference reinforcement learning method to learn the potential field function and determine an appropriate strategy for each agent that can satisfy the requirements of collaborative motion and obstacle avoidance for large-scale UAV swarms. First, a new swarm structure is proposed to provide reserved maneuvering space for UAVs during flight. After encoding the obstacle avoidance behavior of multiple UAVs into spatial graphs, a graph attention mechanism (GAT) was employed to extract the dynamic information from them. Consequently, each individual autonomously generate actions based on its local data. Second, a new distributed control algorithm based on multi-agent reinforcement learning (MARL) is proposed to learn the potential field function from the local information. Each individual can repel and cooperate with the target within a short range and attract objects over a long distance. Finally, simulation results demonstrate the effectiveness and superiority of the proposed method, which has great potential for application in online autonomous collaboration.

Dynamic Screening Strategy Based on Feature Graphs for UAV Object and Group Re-Identification

In contemporary times, owing to the swift advancement of Unmanned Aerial Vehicles (UAVs), there is enormous potential for the use of UAVs to ensure public safety. Most research on capturing images by UAVs mainly focuses on object detection and tracking tasks, but few studies have focused on the UAV object re-identification task. In addition, in the real-world scenarios, objects frequently get together in groups. Therefore, re-identifying UAV objects and groups poses a significant challenge. In this paper, a novel dynamic screening strategy based on feature graphs framework is proposed for UAV object and group re-identification. Specifically, the graph-based feature matching module presented aims to enhance the transmission of group contextual information by using adjacent feature nodes. Additionally, a dynamic screening strategy designed attempts to prune the feature nodes that are not identified as the same group to reduce the impact of noise (other group members but not belonging to this group). Extensive experiments have been conducted on the Road Group, DukeMTMC Group and CUHK-SYSU-Group datasets to validate our framework, revealing superior performance compared to most methods. The Rank-1 on CUHK-SYSU-Group, Road Group and DukeMTMC Group datasets reaches 71.8%, 86.4% and 57.8%, respectively. Meanwhile, our method performance is explored on the UAV datasets of PRAI-1581 and Aerial Image, the infrared datasets of SYSU-MM01 and CM-Group and the NIR dataset of RBG-NIR Scene dataset; the unexpected findings demonstrate the robustness and wide applicability of our method.

DEVELOPMENT OF CONTROL LAWS OF UNMANNED AERIAL VEHICLES FOR PERFORMING GROUP FLIGHT AT THE STRAIGHT-LINE HORIZONTAL FLIGHT STAGE

The article proposes an improved approach to controlling groups of unmanned aerial vehicles (UAVs) aimed at increasing the overall efficiency and flexibility of the control process. The use of a heterogeneous external field, which varies both in magnitude and direction, allows achieving greater adaptability and accuracy in controlling a group of UAVs. A vector field for unmanned aerial vehicles determines the direction and intensity of the vehicles' movement in space. Such vector fields can be used to develop UAV control laws, including determining optimal flight paths, controlling speed, avoiding obstacles, and ensuring coordination of a group of UAVs. The subject of the study is the methods of controlling groups of autonomous UAVs, where each vehicle may have different speeds and flight directions. To solve this problem, various methods of using a heterogeneous field have been developed and proposed. Instead of using a homogeneous field that provides a constant flight speed, a vector field is used that adapts to different conditions and characteristics of the vehicles in the group. This method allows for effective group management, ensuring the necessary coordination and interaction between the vehicles. An analysis of recent research and publications in the field of autonomous system control indicates the feasibility of using machine learning, vector fields, and a large amount of data to successfully coordinate the movement of autonomous systems. These approaches make it possible to create efficient and reliable control systems. The aim of the study is to develop laws for controlling the movement of a group of autonomous unmanned aerial vehicles at the stage of straight-line horizontal flight based on natural analogues to improve the efficiency and reliability of their coordinated movement in different conditions. The main conclusions of the research are that the proposed method of controlling groups of UAVs based on a heterogeneous field can be implemented. It takes into account a variety of vehicle characteristics and environmental conditions that are typical for real-world use scenarios. This work opens up prospects for further improving the management of UAV groups and their use in various fields of activity. The article emphasises the relevance of technology development for autonomous unmanned systems, especially in the context of autonomous transport systems.

Event-triggered consensus control method with communication faults for multi-UAV

This paper investigates the event-triggered consensus for a group of unmanned aerial vehicles (UAVs) with communication faults under the assumption that the position sensors of some individuals are damaged. The objective is to make the UAV group reach consensus in urgent tasks such as obstacle avoidance or evasion. Using the Lyapunov stability theory, sufficient conditions to achieve system consensus are given based on different velocity and position interaction topologies. Considering the limited capabilities of sensors and processors, an event-triggered consensus protocol is adopted to reduce the sampling frequency. Finally, simulation results illustrate the effectiveness of our approach.

Research on Scheme Design and Decision of Multiple Unmanned Aerial Vehicle Cooperation Anti-Submarine Based on Knowledge-Driven Soft Actor-Critic

To enhance the anti-submarine and search capabilities of multiple Unmanned Aerial Vehicle (UAV) groups in complex marine environments, this paper proposes a flexible action-evaluation algorithm known as Knowledge-Driven Soft Actor-Critic (KD-SAC), which can effectively interact with real-time environmental information. KD-SAC is a reinforcement learning algorithm that consists of two main components: UAV Group Search Knowledge Base (UGSKB) and path planning strategy. Firstly, based on the UGSKB, we establish a cooperation search framework that comprises three layers of information models: the data layer provides prior information and fundamental search rules to the system, the knowledge layer enriches search rules and database in continuous searching processes, and the decision layer utilizes above two layers of information models to enable autonomous decision-making by UAVs. Secondly, we propose a rule-based deductive inference return visit (RDIRV) strategy to enhance the knowledge base of search. The core concept of this strategy is to enable UAVs to learn from both successful and unsuccessful experiences, thereby enriching the search rules based on optimal decisions as exemplary cases. This approach can significantly enhance the learning performance of KD-SAC. The subsequent step involves designing an event-based UGSKB calling mechanism at the decision-making level, which calls a template based on the target and current motion. Finally, it uses a punishment function, and is then employed to achieve optimal decision-making for UAV actions and states. The feasibility and superiority of our proposed algorithm are demonstrated through experimental comparisons with alternative methods. The final results demonstrate that the proposed method achieves a success rate of 73.63% in multi-UAV flight path planning within complex environments, surpassing the other three algorithms by 17.27%, 29.88%, and 33.51%, respectively. In addition, the KD-SAC algorithm outperforms the other three algorithms in terms of synergy and average search reward.

Research on Efficient Multiagent Reinforcement Learning for Multiple UAVs’ Distributed Jamming Strategy

To support Unmanned Aerial Vehicle (UAV) joint electromagnetic countermeasure decisions in real time, coordinating multiple UAVs for efficiently jamming distributed hostile radar stations requires complex and highly flexible strategies. However, with the nature of the high complexity dimension and partial observation of the electromagnetic battleground, no such strategy can be generated by pre-coded software or decided by a human commander. In this paper, an initial effort is made to integrate multiagent reinforcement learning, which has been proven to be effective in game strategy generation, into the distributed airborne electromagnetic countermeasures domain. The key idea is to design a training simulator which close to a real electromagnetic countermeasure strategy game, so that we can easily collect huge valuable training data other than in the real battle ground which is sparse and far less than sufficient. In addition, this simulator is able to simulate all the necessary decision factors for multiple UAV coordination, so that multiagents can freely search for their optimal joint strategies with our improved Independent Proximal Policy Optimization (IPPO) learning algorithm which suits the game well. In the last part, a typical domain scenario is built to test, and the use case and experiment results manifest that the design is efficient in coordinating a group of UAVs equipped with lightweight jamming devices. Their coordination strategies are not only capable of handling given jamming tasks for the dynamic jamming of hostile radar stations but also beat expectations. The reinforcement learning algorithm can do some heuristic searches to help the group find the tactical vulnerabilities of the enemies and improve the multiple UAVs’ jamming performance.

MULTI-AGENT MODEL OF GROUP CONTROL POINT OF UNMANNED AIRCRAFT AND UNMANNED GROUND VEHICLE WHEN PERFORMING SPECIAL MISSIONS

Unmanned aerial vehicles (UAVs) are becoming very important platforms for conducting combat operations in the russian-ukrainian war, as they significantly reduce the risk of human life during special and combat missions. The UAV group can be used to perform a wide range of military tasks: destruction of enemy equipment, reconnaissance (monitoring) of the area, construction of aerial networks, etc.
 In the conditions of a change in the tactical situation, countermeasures by the enemy, effective management of the UAV group is impossible without an appropriate management system. The UAV group can include both unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UAVs). The following main tasks are assigned to the management system: determination and coordination of the order of joint actions of autonomous UAV groups; formation of flight (ground) tasks and flight (movement) trajectories of the UAV group; dynamic adjustment of the flight task and the role of each UAV during its use; organization of the communication network between the control center and between UAV groups, etc. The tasks and target functions of management of each of their subsystems are determined. The features of the UAV group control cycle are described.
 The main stages of the synthesis of the control system are considered: the choice of the control method (centralized, hierarchical, decentralized and mixed); selection of algorithms (protocols) and organization of the process of data exchange between UAVs. A functional model of the UAV group management system is proposed. Tasks and main management functions of each of the subsystems are defined. Features of the UAV group management cycle are described. The main stages of the synthesis of the control system are considered: the selection of algorithms (protocols) and the organized process of information exchange between UAVs; variants of formation and swarm models of UAV movement in groups; multi-agent decision-making models for the organization of task performance by the UAV group.

Robust Consensus-based Formation Control of a Group of UAV

Consensus-based formation control (CFC) is one of the most phenomenal formation control methods designed to achieve consensus between any vehicles that using and sharing their position and/or linear velocity with each other in a swarm mission. In this paper, a robust CFC (R-CFC) has been designed and proposed to realize pre-defined formation shapes with a team of unmanned aerial vehicles (UAVs). First, the double integrator dynamics of an UAV is presented. Second, the graph theory is explained briefly to understand any adjacency between UAVs. Then, the proposed R-CFC algorithm has been derived and the stability analysis has been proven via algebraic Riccati stability theory based on Lyapunov stability theorem. After that, the effectiveness of the proposed controller has been tested in real time outdoor tests. The experimental results show that the UAVs have been able to create the desired formation shapes and are less affected by external disturbances such as wind thanks to the proposed control algorithm.

Nexus of Deep Reinforcement Learning and Leader–Follower Approach for AIoT Enabled Aerial Networks

The Industrial Internet of Things (IIoT) is a new industrial 4.0 paradigm that combines IoT, robotics, cyber-physical systems, and other future industrial advancements. Unmanned Aerial Vehicles (UAVs), part of the IIoT infrastructure, have a significant potential for civil and military purposes. Through the Artificial Intelligence of Things (AIoT), a well-organized group of UAVs outperforms a single large UAV in terms of device scalability, maintenance, and expense. Therefore, the UAV swarm with industry 4.0 intelligence can be used for a wide range of 24/7 security and remote monitoring applications. Though multi-UAV systems are beneficial, their application has many challenges. There is a high risk of collision in the multi-UAV system without coordination. This paper proposes an AIoT-based Navigation and Formation Control (AIoT-NFC) mechanism to scale down the collision risk by combining Deep Reinforcement Learning (DRL) with the leader-follower approach. In AIoT-NFC, a Deep Deterministic Policy Gradient (DDPG) based algorithm is proposed to navigate UAVs in remote surveillance without colliding with obstacles and other UAVs. Furthermore, the AIoT-NFC system incorporates a fault tolerance mechanism that can handle the scenario of a leader's failure due to actuator malfunction. Experimental results show that the AIoT-NFC achieves faster convergence with a lower collision rate. AIoT-NFC reduced the collision rate by 14.99% compared to existing navigation methods in successful formation without colliding with the other UAVs.

Two-Hop Cooperative Caching and UAVs Deployment Based on Potential Game

This paper explores the joint cache placement and 3D deployment of Unmanned Aerial Vehicle (UAV) groups, utilizing potential game theory and a two-hop UAV cooperative caching mechanism, which could create a tradeoff between latency and coverage. The proposed scheme consists of three parts: first, the initial 2D location of UAV groups is determined through K-means, with the optimal altitude based on the UAV coverage radius. Second, to balance the transmission delay and coverage, the MOS (Mean Opinion Score) and coverage are designed to evaluate the performance of UAV-assisted networks. Then, the potential game is modeled, which transfers the optimization problem into the maximization of the whole network utility. The locally coupling effect resulting from action changes among UAVs is considered in the design of the potential game utility function. Moreover, a log-linear learning scheme is applied to solve the problem. Finally, the simulation results verify the superiority of the proposed scheme in terms of the achievable transmission delay and coverage performance compared with two other tested schemes. The coverage ratio is close to 100% when the UAV number is 25, and the user number is 150; in addition, this game outperforms the benchmarks when it comes to maximizing MOS of users.

Planning of UAV Flight Routes in the Problems of Group Patrolling of the Extended Territories

Currently one of the promising areas of joint use of unmanned aerial vehicles (UAVs) is group air patrolling of large territories. Here the organization of patrolling assumes the solving the planning problem of routes flight of UAV group. The paper considers the problem of optimal planning of flight routes of the same type of UAVs during group patrolling of large territories. The territorial waters or narrow border areas of any State may serve as an example of such territories. It is suggested that the patrolled area has an elongated shape and is divided into a chain of adjacent patrol zones prescribed by a separate UAV. The drone’s flight route passes through adjacent zones. The flight task performed periodically by each drone consists in moving it to a given flight zone, collecting operational data and transmitting this data to a control point (center, station). The optimization aspect of UAV flight route planning is to minimize the maximum time required to complete flight tasks. The considered problem of group patrolling reduced to the multiple traveling salesman problem — one of the classic intractable combinatorial optimization problems. A brief analysis of modern methods for solving the multiple traveling salesman problem is given. Due to the lack of effective exact methods for solving this problem, it is natural to use approximate heuristic and metaheuristic methods focused on solving NP-hard optimization problems, reducing the full search and giving a solution close to the exact one. The multiple traveling salesman problem considered in this paper is reduced to the problem of integer linear programming, for the solution of which a genetic algorithm implemented in MATLAB based on the mathematical package Global Optimization Toolbox is proposed. An illustrative example of patrolling by three UAVs of an extended territory with 11 adjacent zones is considered. Computational experiments confirm the effectiveness of the algorithmic solutions proposed in the work.

Distribution of tasks in a clustered field of goals for homogeneous and heterogeneous UAV groups

This work is devoted to the distribution of tasks in groups of unmanned aerial vehicles (UAVs) under conditions of a significant excess of the number of tasks over the number of agents. The main tasks solved by UAVs: survey and reconnaissance of territories, detection of dangerous objects or places of emergencies, search for victims, etc. The efficiency of solving the problems listed above achieved by the simultaneous use of a group of UAVs, the ele-ments (agents) of which can carry out the tasks of inspecting and scanning various areas of space in parallel. The article proposes an iterative method for distributing tasks in a group of UAVs with a significant excess of the num-ber of tasks over the number of agents (5-20 times). The proposed method for heterogeneous groups of UAVs based on a two-stage procedure for distributing agents of different specializations among task clusters, taking into account the agent's value function. At the first stage, the base part of the agents is distributed, the remaining agents at the second stage distributed in order to average the distance traveled by each agent. Execution of tasks within a cluster implemented by simulating annealing. To evaluate the effectiveness of the method variants, a comparison made with the greedy task distribution algorithm and the collective goal distribution algorithm. The analogs under consideration are widespread, universal and have a high convergence of the solution. Experimental studies car-ried out by computer simulation, where 2000 experiments carried out with various changes in the number of group agents and generation of a task map. The results showed the high efficiency of the task distribution method in terms of reducing the distance traveled by the agents of the UAV group when performing tasks in comparison with analogues. The efficiency of the path traveled by agents is up to 28% depending on the number of agents and tasks in the cluster, which is a scientific increment of the result of the study.

Review of target objects for the group-operated unmanned aerial vehicles application

Currently, the big number of companies, and specifically in Russia, is engaged in designing unmanned aerial vehicles (UAV). It is caused by the tendency to reduce human labor in various fields of activity, in particular, associated with a great risk to life and health. All these factors lead to the steady growth of different-type missions for the UAV application. Urgency of the research generates considerable interest towards the issues of the UAV application. As part of developing a method for determining UAV design parameters, the author analyzed the results in this domain, considering the operation in a group of aircraft. In order to propose a conceptual aircraft solution for designing a model, the analysis of target objects for the UAV group application was conducted. The article considers various target objects for the application of UAV groups. The result of the analysis for target objects can form the basis for the conclusion involved with the imposed requirements for the design of this UAV class. Compared to other specialists’ articles, this paper is distinguished by synthesizing the achievements in the sphere of group-operated UAV design, as well as by establishing the basic trends for their application while conducting warfare.

Construction of an Adaptive Motion Control System Optimal Information Exchange Scheme for a Group of Unmanned Aerial Vehicles

The paper considers the problem of modeling an adaptive control system information exchange for a group of unmanned aerial vehicles (UAVs). The movement of the UAV group is carried out in accordance with the adaptive algorithm for optimal control. Optimal controls are constructed to provide a minimum of the total energy expended. The parameters of the mathematical model of the movement of the UAV group are refined during the flight in accordance with changing external conditions. In accordance with this, the control actions are specified. This task requires significant computing resources and imposes special requirements on the information exchange system between the UAV and the control point. A scheme of information exchange between the UAV and the control point is proposed, which makes it possible to calculate the optimal parameters of the transmitting devices.

Research on Obstacle Avoidance Control of Multiple UAV Formation based on Genetic Algorithm

The UAV (Unmanned Aerial Vehicle) group cooperative formation flight technology has the advantages of wide coverage, large activity radius, strong overall search ability and high efficiency of the aircraft group. Therefore, it is suitable for various complex tasks in the military field such as battlefield environment reconnaissance, tactical attack and cooperative search. This paper proposes a consistency control strategy based on GA (Genetic Algorithm) and applies it to multi-UAV formation obstacle avoidance, which can effectively solve the collision between UAVs and between UAV formation and obstacles. Demodulate the received ground desired control command, and introduce an additional auxiliary traction acceleration through GA to avoid the local optimal solution. The auxiliary traction acceleration is related to the speed and relative position of UAV and obstacles. It can be used as a disturbance to solve the local optimal solution, and also as an auxiliary acceleration to improve the speed of avoiding moving obstacles. Finally, the rapid formation and obstacle avoidance of UAV fleet during flight are realized, and the survivability of the fleet in the battlefield environment is improved.

Multi-Agent Reinforcement Learning-Based Coordinated Dynamic Task Allocation for Heterogenous UAVs

The coordinated dynamic task allocation (CDTA) problem for heterogeneous unmanned aerial vehicles (UAVs) in the presence of environment uncertainty is studied in this paper. Dynamic task allocation mainly solves the problem of resource reallocation after new tasks appear, so that the multi-UAV systems can quickly respond to further information and objectives. In this paper, the CDTA strategy for heterogenous UAVs is proposed through proposer-responser mechanism and prioritized experience replay, in which the multi-agent reinforcement learning (MARL)-based coordinated network is constructed to propose request, and the Q-network is developed to approximate expected return to determine the responser whether to participate in the dynamic task. The CDTA algorithm considers the uncertainty of dynamic task and has a high scalability in different UAV groups, which can reduce the burden of online calculation and increase the speed of online operation effectively. The experiment proves that the priority experience replay speeds up the convergence of the algorithm, and the scalability of the algorithm is verified within 10-180 UAVs. Comparison simulations with the game theory-based and reinforcement learning-based methods are provided to show the effectiveness of the proposed algorithm.